05:34:10 __uguu__ copypaste: that's a lot of tape
05:39:33 ~copypaste __uguu__: indeed
05:39:44 ~copypaste here's the current problem if anyone cares
05:39:53 ~copypaste all php5 processes go into uninterruptible sleep
05:40:00 ~copypaste until pm.max_children
05:40:09 ~copypaste so if that's 250, 251 [D] php5-fpm processes
05:40:29 __uguu__ oh lord
05:40:35 ~copypaste since you can't kill a D process without rebooting
05:40:38 ~copypaste it fucks up the site
05:40:44 ~copypaste and the php5-fpm manager can't kill it either
05:40:54 ~copypaste i did a ton of `stat` commands to try to find what I/O pipe was broken
05:40:57 ~copypaste i had no luck.
05:41:10 ~copypaste every NFS mount is on
05:41:14 __uguu__ so.... have you tried turning it off then on again? :D
05:41:56 ~copypaste yes
05:42:00 ~copypaste i've done so many times :3
05:42:02 ~copypaste that's the "fix"
05:42:51 __uguu__ try playing nntpchan for a bit maybe srndv2 will feel better on your box than php ^)
05:43:39 ~copypaste good plan
05:43:41 __uguu__ you know how they got distributed denial of service? why aren't there distributed magic mushrooms?
05:43:42 ~copypaste just replace 8ch.net
05:43:46 ~copypaste with an nntp daemon
05:43:59 ~copypaste if anyone complains
05:43:59 ~copypaste say
05:44:02 ~copypaste "it is what it is"
05:44:05 ~copypaste then kill myself
05:44:06 ~copypaste :^)
05:45:04 __uguu__ no don't replace it, just do that beta thing like you did with josh
[...]
08:25:05 Jesus file_get_contents(http://8ch.net/8chan-captcha/entrypoint.php?mode=check&text=fixcne&extra=abcdefghijklmnopqrstuvwxyz&cookie=merpjyaudctmcwzpeaiz): failed to open stream: HTTP request failed! HTTP/1.1 503 Service Unavailable /srv/http/dnsbls_bypass.php:20
08:25:17 __uguu__ yup
08:25:20 __uguu__ diskfail i think
08:40:35 Jesus ;_;
[...]
08:48:52 __uguu__ uhhhh
08:52:14 ~copypaste whole site's down until further notice.
[...]
08:54:52 ~copypaste __uguu__: https://8ch.net/wewlad
[...]
08:56:14 __uguu__ wow that was fast
08:57:19 rebob1 So, is HW just going to reset the server, or is he trying to figure out what's wrong?
08:57:34 * __uguu__ ?\(._.)/? idk
08:59:33 darlene this is why you should always hack docker images and backups ready to spin up
08:59:38 darlene *have
09:00:22 __uguu__ docker doesn't fix this scaling issue
09:00:35 darlene scaling no but coming back up quick sure
09:00:38 darlene is all i'm saying
09:01:04 rebob1 Maybe a HD is failing and the OS isn't handling it gracefully.
09:01:10 darlene probably
09:01:21 darlene but still the only downtime shoudl ever be waiting for dns to migrater
09:06:12 melonstorm a clusterfuck on top of a clusterfuck on top of a clusterfuck won't be easily scalable, docker or not