Abyss Server Statistics

tfh

admin wrote
tfh,
The figures are pretty impressive!
By the way, the screenshots suggest that we'd better pretty print the values and use other units (MB/GB/TB) instead of KB only!
Some percentages and a nice visualization would probably make sense too. :)

I can only agree ;-)
But I really love how stable your software is running :)

pkSML

admin wrote
By the way, the screenshots suggest that we'd better pretty print the values and use other units (MB/GB/TB) instead of KB only!

Sorry to drop in on this thread a month later... Haven't been to the forum in a little while.
Aprelium, I would suggest using raw values in the output and use JavaScript to convert them in the page display. That way you could view the source code of the page and get the actual byte numbers, as well as have scripts grab the precise info. But JS could translate it for simple understanding in the browser (i.e. KB, MB, GB, TB...).
And tfh, I'm curious what hardware you're running your server on. Your uptime is amazing! Are you using ECC RAM? If not, a reboot might be in order ;)

tfh

pkSML wrote

admin wrote
By the way, the screenshots suggest that we'd better pretty print the values and use other units (MB/GB/TB) instead of KB only!

Sorry to drop in on this thread a month later... Haven't been to the forum in a little while.
Aprelium, I would suggest using raw values in the output and use JavaScript to convert them in the page display. That way you could view the source code of the page and get the actual byte numbers, as well as have scripts grab the precise info. But JS could translate it for simple understanding in the browser (i.e. KB, MB, GB, TB...).
And tfh, I'm curious what hardware you're running your server on. Your uptime is amazing! Are you using ECC RAM? If not, a reboot might be in order ;)

The machine is actually a PC I built myself, based on an AMD Ryzen 5 Pro 4650. But I think you are looking at the uptime (which is just the total uptime since I installed Abyss on this PC) instead of the uptime since the last restart. Only the Windows updates require a restart once in a while. But even installing those updates only take a short amount of time according to the uptime robot website:

pkSML

tfh wrote
But I think you are looking at the uptime (which is just the total uptime since I installed Abyss on this PC) instead of the uptime since the last restart.

Yes, you're right. Silly me. But it is impressive how much data has been served by your Abyss instance!
I just checked and my total uptime is 1614 days and the transferred data is 1278583694 KB (1.2 TB).

tfh

pkSML wrote
Yes, you're right. Silly me. But it is impressive how much data has been served by your Abyss instance!
I just checked and my total uptime is 1614 days and the transferred data is 1278583694 KB (1.2 TB).

Yep. That's my traffic every month ;-)

tfh

tfh wrote

Running 2 domains, about 10 sub-domains. Moved to X2 a bit over a year ago and VERY VERY happy with the quality of stability of Abyss Webserver. And not to mention the support from Aprlium.
Almost 10 TByte of data transfered.

A bit over a year down the road:

In the mean time 4 domains, about 15 sub-domains. Over 30 TByte of data served, so around 20TByte during last year.
A bit less happy with the stability these days. Already filed some bugs quite some time ago, but it kind of remains quiet. I'll should e-mail support I guess.

admin

tfh wrote
A bit less happy with the stability these days. Already filed some bugs quite some time ago, but it kind of remains quiet. I'll should e-mail support I guess.

As we discussed by email, the 20-second blackout issues seem to be caused by a visitor "stealing" all your bandwidth and saturating all your connections.

tfh

admin wrote

tfh wrote
A bit less happy with the stability these days. Already filed some bugs quite some time ago, but it kind of remains quiet. I'll should e-mail support I guess.

As we discussed by email, the 20-second blackout issues seem to be caused by a visitor "stealing" all your bandwidth and saturating all your connections.

I actually doubt that's the case.
That user was opening about 20 connection per minute and downloading at a rate of 40mbps. The server has a dedicated 1gbps connection. The server is set to accept 600 simultanious connections.

rayleech

admin wrote
As we discussed by email, the 20-second blackout issues seem to be caused by a visitor "stealing" all your bandwidth and saturating all your connections.

Admin: Could you discuss this particular "blackout" problem? I've had very infrequent reports from a client where a web request takes 15 to 20 seconds to respond, but I've not been able to find a root cause or reproduce.
For the external users, I chalked it up to Internet issues, but there were reports from internal users who never leave the local network structure.
We've checked logs from firewall, switch, abyss, event log, etc., and found nothing that seemed to apply. So far, the ONLY thing I've found is one instance where one log record appears out-of-order, but a quick look at application log showed the longest transaction was 2.25 seconds. Couldn't find anything else that would have sucked up resources (like virus scan, etc.)
I've sanitized the weird log records below.
67.01.02.003 - - [14/Mar/2023:21:51:41 -0400] "GET /cgi-bin/app.cgi?sys=999.system HTTP/2.0" 200 177773 "http://www.ourdomain.com/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/111.0.0.0 Safari/537.36"
((48 other events: mixed static and cgi requests))
66.111.22.333 - - [14/Mar/2023:21:52:00 -0400] "GET /.well-known/traffic-advice HTTP/1.1" 302 5 "" "Chrome Privacy Preserving Prefetch Proxy"
67.01.02.003 - - [14/Mar/2023:21:51:42 -0400] "GET /cgi-bin/app.cgi?sys=999.system HTTP/2.0" 200 177773 "http://www.ourdomain.com/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/111.0.0.0 Safari/537.36"

tfh

I sometimes even have the problem when accessing it on the local machine itself. So the root of the problem is not physical network related.
It seems that sometimes a thread hangs and because of that does't respons to any requests anymore until there is a time-out.
I also notice that times the memory consumption grows to over 200MByte, while usually Abyss only takes 40 - 50MByte. So it looks like there is a memory lead somewhere as well.

rayleech

tfh:
My issue appears to be a little different. The few random reports I've gotten don't appear to time-out. They "eventually" display the full response.

tfh

rayleech wrote
tfh:
My issue appears to be a little different. The few random reports I've gotten don't appear to time-out. They "eventually" display the full response.

Well, that's also what I mean....
The time-out setting for a cennection in Abyss is 20 seconds. So after that 20 second wait, you finally get the page that was requested. Sorry for my incomplete description.

rayleech

tfh: I was finally able to capture enough information to generate a support ticket. Hopefully, this will lead to some info.
The user reports were never specific enough, so I created a sensor that runs on the same server that hosts the web site. Every five seconds, I hit the site locally and retrieve a blank html file. I log success/failure of each attempt, and the timestamps line up because it's the same server.
I was able to catch one outage yesterday where a static file was out of order by 2 minutes 52 seconds, something that normally responds same second.
I doubt the access.log is going to be enough to identify the problem. Hoping there's some kind of debug log that can be turned on for more info.

tfh

rayleech wrote
tfh: I was finally able to capture enough information to generate a support ticket. Hopefully, this will lead to some info.
The user reports were never specific enough, so I created a sensor that runs on the same server that hosts the web site. Every five seconds, I hit the site locally and retrieve a blank html file. I log success/failure of each attempt, and the timestamps line up because it's the same server.
I was able to catch one outage yesterday where a static file was out of order by 2 minutes 52 seconds, something that normally responds same second.
I doubt the access.log is going to be enough to identify the problem. Hoping there's some kind of debug log that can be turned on for more info.

Well, all information will probably be welcomed by the Apprelium software guys.
Thanks for the time you're also investing in this.

rayleech

It appears the logs were enough (phew). There appears to be a bug in the HTTP/2 module, so we've disabled HTTP/2 on the server that responds weirdly.
tfh: They also said to stop using the statistics section in the console. There can be a concurrency issue that could cause a problem.
They are working on the HTTP/2 issue now.

tfh

rayleech wrote
It appears the logs were enough (phew). There appears to be a bug in the HTTP/2 module, so we've disabled HTTP/2 on the server that responds weirdly.
tfh: They also said to stop using the statistics section in the console. There can be a concurrency issue that could cause a problem.
They are working on the HTTP/2 issue now.

Hi,
Thanks for the update.
As for the statistics section in the console: Yes, I am aware of that. I already informed them about those issues well over a year ago ;-)

admin

Many thanks for all of you for the logs and the reports. Hopefully we'll get past that issue and fix it.

tfh

Time for a little celebration:

admin

tfh wrote
Time for a little celebration:

That's quite a milestone. Congrats!

tfh

In the meantime started using a new PC as a webserver, so... the stats have been reset:

« Previous Page