Logs for #pizarro

Go to: #trilema #pizarro #asciilifeform #trilema-mod6 #chainstate #eulora

2019-9-30

lobbes_field: BingoBoingo: Loox like auctionbot went down. I'm stuck in the mines and have some time sensitive errands to run afterwards, so won't be back to terminal until later tonight.
lobbes_field: I can extend the auction for the downtime. Will make note not to charge for downtime hours as well (even though auction billing is still in suspended state)
BingoBoingo: lobbes_field: ty, please do
lobbes_field: Yw, and sorry for teh inconvenience
thimbronion: BingoBoingo: I'm tracking down an issue with html in comments on my blog. The dialog around here: http://logs.ossasepia.com/log/trilema/2019-03-21#1903838 seems related, but all the p.bvulpes pastes are gone, as is the cascadianhacker blog with a related post. Do you know what I need to do to fix this issue?
ossabot: Logged on 2019-03-21 05:39:42 mircea_popescu: 4.x likely.
BingoBoingo: thimbronion: http://btcbase.org/patches/mp-wp_html-comments-regrind is where I usually start looking
thimbronion: BingoBoingo: thanks
thimbronion: BingoBoingo: Is there a way I could get tmux installed?
thimbronion: Or do people generally just download the src and build for themselves whatever they need?
auctionbot: --- end of auction list, 161mn total bids ---
auctionbot: B#1061 O=161mn LB=161mn E=2019-10-02 05:55:34.297573 (5h55) >>> 1209 Wired Filthy Fiats, WU is fine but not preferred
auctionbot: --- end of auction list, 161mn total bids ---
jurov: !Xbid 1061 155mn
lobbes: BingoBoingo: seeing as how auctionbot went down *again* I'm going to just give you this entire auction on-the-house.
lobbes: auction was supposed to end today at roughly 3:00 pm (15:00) ET. If you'd like, I can extend auction so it closes at that time tomorrow; give last minute people a chance to bid at around the same timeframe
BingoBoingo: lobbes: Aite, see also http://logs.nosuchlabs.com/log/trilema/2019-10-02#1939671
snsabot: (trilema) 2019-10-02 BingoBoingo: asciilifeform: Issue with an upstream switch is the report.
lobbes: yeah, they supposedly fixed it though right? Plus I'ma be working from home tomorrow, so if any further outages I can bring it back online ASAP.
lobbes: bot really should, imho, bring itself back online, so this is still a failure on me in my book
lobbes will stop by castle of spyked to see how he handles feedbot in such instances
auctionbot: B#1061 O=161mn LB=151mn E=2019-10-03 05:55:34.297573 (5h55) >>> 1209 Wired Filthy Fiats, WU is fine but not preferred
auctionbot: --- end of auction list, 151mn total bids ---
bvt: BingoBoingo: should i reuse the information from the last auction for sending the wire, or are there any changes?
BingoBoingo: bvt: The same information works
bvt: BingoBoingo: sent
BingoBoingo: ty, please invoice at will
hobotrinque: pipes are clogged again, ping to 161.0.121.240 is failing.
hobotrinque: asciilifeform ^
hobotrinque: BingoBoingo is the dc down?
asciilifeform: it is
lobbes_field: back again, but for how long..
asciilifeform: for 10s seems.
hobotrinque: cannot ping from singapore either ftr
asciilifeform: hobotrinque: whole dc unpiped atm.
asciilifeform: loox like alive again. for nao.
BingoBoingo: Deedbot hasn't given me a challenge to decrypt for voice yet. Datacenter has been contacted
asciilifeform: got it
mod6: Thank you!
lobbes_field: damn. down again
BingoBoingo: ATTN: If you are running an IRC bouncer on Pizarro, please configure it to apply your freenode cloak BEFORE joining channels
lobbes_dumbphone: Report from saltmines viewport: DDos widespread. Both my VPSen are down  as well (logs.minigame.biz and lobbesblog.com).
lobbes_dumbphone: Looks like latter back up meanwhile, but this be frustrating.
BingoBoingo: lobbes_dumbphone: The strongest suspicion is connecting to IRC without network cloaks is what drew the attack.
lobbes_dumbphone: Seems like it. I'ma move to reconfigure bouncers to cloak before joining later tonight. Still, I've so many bots; will need a cloak for each one...
BingoBoingo: lobbes_dumbphone: You appear to have some sort of bot running on my rockchip. How would you like me to keep it from reconencting to IRC while I try to get things online again?
lobbes_dumbphone: BingoBoingo: that is my irssi bouncer. Recall I'm using your rk as a stand-in for shared hosting
lobbes_dumbphone: Feel free to just kill the process
BingoBoingo: lobbes_dumbphone: Aite
lobbes_dumbphone must descend back into mines. Will be monitoring logs
thimbronion: fwiw my bouncer (running on non-pizarro box) was disconnected yesterday, and a few days prior, and although I have a cloak, it does not get applied before bouncer rejoins channels.
thimbronion: Currently not experiencing connectivity issues.
dorion: http://logs.ossasepia.com/log/trilema/2019-10-04#1940135 << BingoBoingo I am below this range. I've not run IRC from there, but have been setting up my blog (http://dorion-mode.com/) over the past week, preparing to report FG
ossabot: (trilema) 2019-10-04 BingoBoingo: asciilifeform: "The attacks are moving on the range. It started with .97, then move to .99, then to .200 and so on." This supports lazy attack targeting uncloaked irc connections.
dorion: test results and well as trb on rk status.
asciilifeform: dorion: http://dorion-mode.com dun load...
dorion: asciilifeform: not right now because entire pizarro /24 is down, right?
asciilifeform: correct
dorion: therefore, reporting that the IP for dorion-mode.com was/is public in DNS records and not included in the range. I don't know how much this information actually helps, but it's information I have to share.
asciilifeform: ty dorion . might prove relevant.
bvt: BingoBoingo: i have created the invoice
BingoBoingo: bvt: ty
BingoBoingo: lobbes: You should be able to ssh through the shared server to access your rockchip and reconfigure what's running there
BingoBoingo: !Qlater tell billymg Please send my an SSH public key. I will set up an account on the shared server for you from which you can access your Rockchip
lobbesbot: BingoBoingo: The operation succeeded.
lobbes: BingoBoingo, roger that
lobbes: BingoBoingo, no dice. It is your rk that I use for the shared hosting (can't recall the reason atm). Can't even ping that box atm
lobbes: BingoBoingo, nvm
BingoBoingo: lobbes: make sure you use -p 50022 you do have an account on the shared box
lobbes: my old key for the shared server still worx!
lobbes: BingoBoingo: okay, auctionbot has been stopped. Should be safe to bring back without it auto-joining
BingoBoingo: lobbes: iirc you had something running on the .197 rockchip as well?
lobbes: BingoBoingo, the .197 rk was the one I just logged into (the one with auctionbot). I thought you had killed my irssi process on your box earlier, though I can check that now
BingoBoingo: lobbes: I killed the one on my rockchip. I could ssh into it from the rack.
lobbes: okay, then we ought to be good. I didn't have anything running on the shared server
BingoBoingo: lobbes with respect to the shared machine http://logs.ossasepia.com/log/trilema/2019-10-04#1940199
ossabot: (trilema) 2019-10-04 BingoBoingo: I have stopped irissi, weechat, and znc services on the shared machine. I am about to request the portion of the IP space not known to be running uncloaked bouncers is restored shortly.
lobbes: kk, then we are solid it seems
lobbes: looks like another small pipe outage occurred (lasted ~1 min)
BingoBoingo: More of a clog
mod6: asciilifeform: The tbf box in the rack would be nice to be backed up -- I believe, from memory, this is a raid configuration in there. If the drives are pulled and not numbered correctly, probably can't re-create.
asciilifeform: mod6: in #t plz
bvt: BingoBoingo: did you receive the last wire?
BingoBoingo: bvt: It hit today, will get your invoice taken care of
BingoBoingo: bvt paid
PeterL: So, bingoboingo, it looks like from #trilema log that Pizarro is not coming back online?
asciilifeform: PeterL: see #trilema from past 48h.
asciilifeform: http://logs.ossasepia.com , http://logs.ericbenevides.com/log , still work.
PeterL reading now
BingoBoingo: PeterL: Our datacenter by all appearances is dying, there seems to be no prudent way to continue in Uruguay.
BingoBoingo: As an ISP
bvt: BingoBoingo: ty, i started to worry that the wire got stuck somewhere between banks
BingoBoingo: bvt the incoming international ones take a bit of time, but almost always under a week
bvt: BingoBoingo: would it be possible to get the my data from shared hosting (homedir + dbdump) as well? if you send the ssh key, i will set up an account for rsyncing/scping data.
BingoBoingo: bvt: Working on baking the .tar.gz.gpg files. Will be uploading to the deebot server with many thanks to trinque.
bvt: ok, ty
thimbronion: BingoBoingo: backup retrieved - thanks.
BingoBoingo: thimbronion: No, thank you
BingoBoingo: http://logs.ossasepia.com/log/trilema/2019-10-09#1944373 << If any shared hosting customers haven't been following #trilema http://logs.ossasepia.com/log/trilema/2019-10-09#1944373
ossabot: (trilema) 2019-10-09 trinque: bvt, cruciform, danielpbarron, hanbot, lobbes, mats, mocky, mod6, nicoleci, peterl, thimbronion, whaack : http://paste.deedbot.org/?id=LxFh
ossabot: (trilema) 2019-10-09 trinque: bvt, cruciform, danielpbarron, hanbot, lobbes, mats, mocky, mod6, nicoleci, peterl, thimbronion, whaack : http://paste.deedbot.org/?id=LxFh
bvt: ty, i have donwloaded the tgz as soon as it went online
BingoBoingo: bvt: ty for confirming
mod6 is grabbing as well.
whaack: BingoBoingo: also ack that i received backup and have successfully restored ztkfg.com
BingoBoingo: ATTN http://logs.ossasepia.com/log/trilema/2019-10-10#1944673
ossabot: (trilema) 2019-10-10 BingoBoingo: ATTN ALL: Latecho has silently disabled my access to the datacenter. All that remains in the rack per my last survey are 1. Switch for the incoming connection 2. less sophisticated switch serving RK plant 3. RK plant (minus drives) 4. Rails 5. A small mountain of network and power cables 6. Many Racknuts, mostly unsused 7. One platic tote box, empty
BingoBoingo: !Qlater tell peterl were you wanting your blog set up on the 'anyserver'?
lobbesbot: BingoBoingo: The operation succeeded.
PeterL: hi bingoboingo, it sounds like you are ready for blogs to get set back up?
lobbesbot: PeterL: Sent 14 hours and 17 minutes ago: <BingoBoingo> were you wanting your blog set up on the 'anyserver'?
PeterL: I don't know if you said yet, where is 'anyserver' located?
BingoBoingo: PeterL: Located in the US. If you want I can stand your blog up sometime this evening after making it out to do some errands.
PeterL: Yes, please.
BingoBoingo: PeterL: ty. Note that I intend for 'anyserver' to be an interim solution to hosting blogs. The fact I can't touch it like UY1 means its materially different enough that I do want folks explicitly requesting interim hosting on the 'anyserver'
PeterL: If asciilifeform ends up setting up a rack in his neighborhood, would this migrate there?
BingoBoingo: I'm willing to help set up a shared hosting environment there after publishing my recipe, if requested. If 'anyserver' ends up not sucking I may keep it around for the sake of not hanging everything off the same line and keep it for my use.
asciilifeform: PeterL: what gets migrated to asciilifeform's rack, is 100% up to the inhabitants
PeterL: Are you running both places as Pizarro, or would his rack be something separate?
asciilifeform: PeterL: see running #t thread re this.
PeterL: yes, I am reading todays log atm
PeterL: iirc pizarro is holding a bunch of FGs. Out of curiosity, if you were to sell them, how much would you ask for one?
asciilifeform: PeterL: if indeed this, then i expect -- no less than what was paid. but i dun expect they'll be sold on open market, no. (see again #t thrd.)
PeterL: are you (NSA) planning to make another run of them?
asciilifeform: PeterL: there's no current plan for this.
danielpbarron: PeterL, I have FGs. Make me an offer.
asciilifeform: wb danielpbarron
BingoBoingo: dorion: As Pizarro is getting parted up, would you like me to take any backups from your Rockchip's drive?
BingoBoingo: !Qlater tell PeterL http://paste.deedbot.org/?id=Tqmp
lobbesbot: BingoBoingo: The operation succeeded.
PeterL: BingoBoingo: ack on the gpggram ... when I go to the IP you gave me it just shows your blog, what am I missing?
lobbesbot: PeterL: Sent 3 hours and 2 minutes ago: <BingoBoingo> http://paste.deedbot.org/?id=Tqmp
BingoBoingo: PeterL: Did you go to your domain yet?
PeterL: it doesn't seem to load
BingoBoingo: Do you have an entry for the old ip in your hosts file?
PeterL: hmm, maybe, let me check
PeterL: aha, now it works! thanks
PeterL: wp-admin doesn't seem to load though
BingoBoingo: PeterL: You need to shell in and edit your .htaccess files
PeterL: "Internal Server Error"
BingoBoingo: Right, no https mean you gotta spec an allowed IP address or several in the .htaccess in /wp-admin and /wp-includes/js
PeterL: did you say something about using a different format for the htaccess file?
BingoBoingo: PeterL: This is apache 2.2 instead of 2.4, 2.4 needed Require statements for IPs. 2.2 lets you use ALLOW and DENY rules
PeterL: it looks like it was just denying everybody
BingoBoingo: PeterL: Well, gotta have a safe default
PeterL: it says I don't have permission to access the wwwlogs directory?
dorion: http://logs.ericbenevides.com/log/pizarro/2019-10-15#1000349 << thanks. is there an opportunity to acquire the setup I rented in part or whole ? i.e. rk + ssd + fg ?
ericbot: Logged on 2019-10-15 21:55:33 BingoBoingo: dorion: As Pizarro is getting parted up, would you like me to take any backups from your Rockchip's drive?
BingoBoingo: dorion: Your drive and the FG are safely in a location I control. The Rockchips are still in the rack to which I do not have access.
BingoBoingo: dorion: Which part of the world would I be shipping to?
dorion: BingoBoingo ok, the Rockchip is perhaps the easiest to replace. last I read the dc wrote they'd allow you to evacuate what remains, correct?
dorion: BingoBoingo I'm in Panama.
BingoBoingo: dorion: The how is contentious.
PeterL: http://peterl.xyz/2019/10/back-up-again/ << hey look, it works
BingoBoingo: PeterL: ty, for confirming.
dorion: BingoBoingo in the short term then I'd like a backup of everything except the /proc /sys /usr/portage/distfiles and /home/rmd/fg directories. as the pizarro parting proceed, I'd be interested in acquiring the gear.
BingoBoingo: dorion: Alright, keep in chan and as we work up how the parting is going to work out I'll make sure they news hits the log hee.
BingoBoingo: here
dorion: BingoBoingo thank you, I will.
BingoBoingo: http://bingology.net/2019/10/16/reasonable-lamp-setup-for-mp-wp-on-centos-6-and-anyserver-terms-of-use/
bvt: BingoBoingo: thank you for the anyserver account (acking receipt of gpg-msg) and the writeup
lobbesbot: bvt: Sent 21 hours and 20 minutes ago: <BingoBoingo> http://paste.deedbot.org/?id=_klh
BingoBoingo: bvt: ty

2019-9-30