Logs for #trilema-mod6

Go to: #trilema #pizarro #asciilifeform #trilema-mod6 #chainstate #eulora

This is the log of #trilema-mod6 : Contained within are the daily on-goings of The Bitcoin Foundation's (http://thebitcoin.foundation) Reference Implementation (trb) development. Do not, under any circumstances, use any code/vpatch/logic/configurations/ideas or otherwise found in this forum. In fact, you should NOT read this at all. Failure to heed these warnings may result in death or dismemberment! ALL ANNOUNCEMENTS, BUSINESS, AND/OR QUESTIONS INVOLVING THE BITCOIN FOUNDATION AND/OR THE BITCOIN REFERENCE IMPLEMENTATION (TRB) WILL TAKE PLACE IN #TRILEMA: YOU HAVE BEEN WARNED.

2016-11-30 | 2017-1-1

ben_vulpes: logs begin from here
asciilifeform: ben_vulpes: they are on www? where ?
ben_vulpes: asciilifeform: shortly, at logs.bvulpes.com
ben_vulpes: where shortly is sometime this week
asciilifeform: neato
ben_vulpes: i still have a bit of work to do in capturing and rendering irc ACTION lines correctly
ben_vulpes: and then i probably have to replicate phf and Framdragger's unicode feats
mod6: sup
mod6: it was -6 this morning.
mod6: supposed to be -19 on saturday
mod6: ok. so, i've got some steps written out to test this thing.
mod6: so im gonna try 'em out and see how it goes.
mod6: and by "this thing" i mean the edge case for beginHeight we've been talkin about
mod6: shinohai: how's the re-sync goin?
shinohai: slowly but good so far :)
mod6: werd
ben_vulpes: asciilifeform: http://logs.bvulpes.com/trilema-mod6?d=2016-12-13#891265d9-349d-447f-8cb4-acb3617e083b
ben_vulpes: all of a sudden those urls look very long
mod6: you could save 8 chars by doing mod6.logs.bvulpes.com
mod6: 7
mod6: damnit. i fucked up that test.
mod6: i'll do it over again tomorrow, ran out of time.
ben_vulpes: yeah, but i don't particularly want to special case every irc channel i stick the bot in.
mod6: aight.
mod6: i should have been thinking about what i was doing.
mod6: smh
ben_vulpes: what went wrong?
mod6: i started scanning from block 10, instead of 10 blocks back.
ben_vulpes: brutal :(
mod6: well... it'll now pick up both txs instead of just the one.
mod6: anyway, w/e. will do it again tomorrow.
ben_vulpes: glad you're putting this thing through the ringer regardless
ben_vulpes: mod6: i think i'll twiddle the log-line html id's to increment from zero, then i can have very short references for each day
ben_vulpes: "on the nth line of the zth day, mircea_popescu did write..."
mod6: werd up mimisbrunnr
mod6: evening
mod6: re-test underway...
shinohai: \o/
mod6: and we have replication of what we thought it would do.
mod6: http://btc.blockr.io/address/info/1KixqueTX751rs12G6mFG3MjMYVpkg9oFq
mod6: sent 0.0005 to 1KixqueTX751rs12G6mFG3MjMYVpkg9oFq @ block 443516
mod6: waited 6 blocks
mod6: sent 0.0001 to 1KixqueTX751rs12G6mFG3MjMYVpkg9oFq @ block 443523
mod6: imported private key with beginHeight set to 443523
mod6: balance was incremented by 0.0001
mod6: now, in the edge case we discussed, there was a step in the middle that I skipped over for this test; sending half of the coins off to a secondary address before the 2nd tx was incoming to the address.
mod6: however, i think this illustrates at least part of this.
mod6: now the last step here is to -rescan and ensure that the balance (from what it is now) goes up by the remaining 0.0005
mod6: will run and report back.
mod6: tomorrow, i'll run another test with that other step inbetween.
mod6: Rescanning last 443528 blocks (from block 0)...
mod6: ok rescan worked. no have additional 0.0005 added to my overall balance.
shinohai: O.o
mod6: *now
mod6: well. so I was going to run that test with the inbetween step like I mentioned yesterday.
mod6: but I need two fully sync'd wallets for that.
mod6: so if you wanna help me out, let me know.
mod6: <+shinohai> O.o << were you confused about the test or results?
ben_vulpes: mod6: why do you need 2 synced wallets?
ben_vulpes: also, if you move the wallet away, trb should generate a new, up-to-date one
ben_vulpes: at least that's what i thought it did.
mod6: well, let's talk through it
mod6: so
mod6: STEP 1: Send 0.0005 to wallet A
mod6: STEP 2: Wait 6 blocks
mod6: STEP 3: Send 0.00025 to some other place (no longer needed)
mod6: STEP 4: Wait 6 blocks
mod6: STEP 5: Send 0.0001 to wallet A.
mod6: STEP 6: Wait 6 blocks
mod6: STEP 7: Import privkey of pubkey address on Wallet A into Wallet B.
mod6: if you don't have two hotwallets, you can't do STEP 3.
mod6: for lastnights test, for instance, I didn't do that step, and used a paper wallet. which is fine for that test.
mod6: but if you want to ~send~ need a separate wallet.
ben_vulpes: step three sends from wallet a?
mod6: Yes, Sir.
mod6: the idea is, that after you scan with a beginHeight of HEAD-6 you should get your orig balance + 0.0001 (in wallet B)
ben_vulpes: less 0.00025?
mod6: and you won't get 0.00035
mod6: ya.
mod6: now lastnight, it worked exactly like this.
mod6: i got the 0.0001, but not the 0.0005 until I did a rescan
mod6: and the same should still hold true here, where i don't get the remaining 0.00025 from that uxto until rescan
ben_vulpes: if you're sending the 0.00025 from A to some other address, it shouldn't show up in the wallet at all, right?
mod6: sorry, busy.
mod6: but.. so step three
mod6: STEP 3: Send 0.00025 (of 0.0005 balance) from wallet A, to wallet X.
mod6: this will leave a uxto of 0.00025 remaining to be claimed for that pubkeyscript in wallet A.
mod6: what we're looking for here is to gather up the remaining balance.
mod6: does this make sense?
ben_vulpes: yes
ben_vulpes: you'll want a final balance after importing the A privkey of 0.0035, right?
mod6: i think once I do this (and i encourage all of you to do the same) it'll hold true to lastnights test and make sense.
mod6: ben_vulpes: exactly.
ben_vulpes: yeah, i gotcha
ben_vulpes: to get back to the boring shit of the test protocol, you should be able to get a fresh, up-to-date (eg "synced") wallet by restarting bitcoind without a wallet.dat in place
mod6: i guess i could do that.
mod6: just do all the gyrations with wallet A with wallet.dat-A, then dump the privkey, then re-start with out a .bitcoin/wallet.dat in place, then do an import
ben_vulpes: sounds correct
ben_vulpes: man, routing around the change address behavior and wallet braindamage is excruciating
ben_vulpes: i'm off for a bit, but i hear what you're saying.
mod6: aight, have fun :]
ben_vulpes: i have to spend some time tooling up to be ready to run these tests as well
mod6: yeah no worries.
mod6: im not in any big rush to send this out or anything.
mod6: it does make life a bit faster ... but not even sure that this feature is a Good Thing (tm)
ben_vulpes: oh i do
ben_vulpes: the alternative is a solipsistic test network
ben_vulpes: which wouldn't even be that bad of a thing to set up
mod6: i could see using the one that exists in the ML today as "defacto", and this one just remaining as a "use this if you really want it".
mod6: anyway, i'll probably feel better about this one after a ton of testing.
mod6: i'd also like to get something better figured out for the command line params.
mod6: i dont like how it is currently setup
asciilifeform: ben_vulpes: fwiw i use 'solipsistic test network'
ben_vulpes: then i suppose i've no excuse to not set one up
shinohai: 14:55 @mod6 <+shinohai> O.o << were you confused about the test or results? <<< I thought for a sec it didn't see but it did great!
mod6: ah, werd :]
mod6: werd
mod6: how can i help
davout: i had to change a couple of things in v.pl to get it to work on my system
mod6: you shouldn't have to change anything in there what so ever.
davout: namely the fact that it relies on matching hardcoded GPG output strings
davout: which aren't in engrish on my system :D
mod6: ooh.
mod6: yeah, engrish only.
davout: if there's interest i'll try to come up with a way to make it handle this properly
mod6: were you able to press v0.5.4 successfully after you made the changes?
mod6: i have no interest in supporting more than just 7bit ascii
davout: no, i had another error, don't have the message at hand here, will post later
davout: 7bit ascii but different language ?
mod6: you had another error from the press command?
davout: yeah
davout: but that's something else
davout: at least output a sensible error message or warning when the language isn't english
mod6: did you try to press more than once into the same directory? it's best, if you have a problem, to clear the press dir away and start fresh
davout: i'll try that
davout: and let you know once i get a chance to retry
mod6: if you end up with a ~/.gnupgtmleft-over because of a fatal error, just clear it away.
mod6: ok please do, and im sure we'll get you up and running.
mod6: if all else fails, we'll get you running a engrish version of gnupg
mod6: <+davout> 7bit ascii but different language ? << ok if it's in the 7bit ascii range is one thing... however, chasing multi-lang support is just gonna be too time consuming for me.
davout: yeah sure, i was simply thinking of maybe finding a easy way on not depending on langs at all
davout: i'll have a look, to each his own itch!11
mod6: for instance, with french, even if you can get by without the c with cedilla, it's still a lot of work to get all of the parsing correct and tested etc.
mod6: <+davout> yeah sure, i was simply thinking of maybe finding a easy way on not depending on langs at all << this would be neat if it's possible.
mod6: let me know if you come up with something there in that vein.
mod6: salud :]
mod6: hang out, and when you get a chance, paste me your errors. we'll get you off and running.
mod6: <@mod6> if you end up with a ~/.gnupgtmleft-over because of a fatal error, just clear it away. << * ~/.gnupgtmp
davout: sure
davout: i understand you also have cucumber tests for v?
mod6: i do. however, they're horribly out of date :/
davout: okay, maybe i'll ask for them later, might feel inclined to re-implement v in ruby for education and trollage purposes :D
mod6: i think at the point of the 99995 release of v.pl, they were all working. now, i'm sure that since the flow has changed dramatically, some tests will fail.
mod6: but they're out there if you wanna look at 'em
davout: ok
davout: see for example, to not depending on the language i changed
davout: $fp = $1 if $r =~ /Primary key fingerprint: (.*)/; $fp =~ s/\s+//g;
davout: to
davout: $fp = $1 if $r =~ /([A-F0-9\s]{50})/; $fp =~ s/\s+//g;
davout: aka. match the FP directly
davout: doing it for the rest of the strings wasn't as straightforward, so i'll need to think a bit more
mod6: yeah, it's nasty because with the way that gpg will output that stuff to stdout, it all comes at you in one blob.
mod6: so there is actually some great care in ensuring that I've got the correct values parsed out.
mod6: s/blob/char stream/
mod6: i'll have to admit that the cucumber tests are pretty neat, and helped me to test v as I was actively developing it... but the tests are not my finest work at all.
mod6: and infact the tests are quite brittle and intra-dependant.
mod6: well some of them are atomic. but the more elaborate the test, the harder time I had making them atomic.
mod6: but, the perl impl. of cucumber is simply an unsupported hack. if I was any good at ruby, and written them in that, i'd probably have something better.
mod6: *shrug*
mod6: anyway, someday i'll probably go back and re-write that stuff
davout: having tests at all is incredibly valuable, brittle tests are much, much better than no tests at all
mod6: fair enough, Sir.
mod6: I'm gonna get some sleep; talk to you a bit later :]
davout: bye!
mod6: mornin'
shinohai: heya mod6 o/
mod6: how goes?
shinohai: Not bad here, getting things sorted and you?
shinohai: tgif
mod6: yeah. this week has been loooooong
mod6: otherwise, just gonna try to stay in this weekend.
mod6: supposed to be -24 tomorrow
shinohai: O.o
shinohai: Same here, got to tie up a few loose ends and gonna stay in
mod6: i've got a few errands to run today though
shinohai: off from work?
mod6: but yeah, trying to stay in. -24 below is where any water in the air turns to ice instead of snow and rains down a fine mist of ice-crystals
mod6: im wfh today. im like "f this shit."
shinohai: >.<
shinohai: Can't blame ya there
mod6: not because of the cold, but because we're gonna get like 5-8" of snow and driving to the train and back will be a nightmare
mod6: the pattern seems to be: 1) Snow for 36-48 hours, 2) Drop tempature by 20-30 degress next day.
mod6: once the clouds lift, all the heat escapes into the atm
shinohai: I'm lucky it's onlt 17 here xD
shinohai: *only
mod6: its actually 9 above right now & snowing.
mod6: HEAT WAVE! where are the girls with their rollerblades & bikinis?
shinohai: :D
mod6: thats my favorite part of spring time around here.
mod6: once it gets up to like 38
mod6: 45 for sure.
shinohai: Spring is like 1 week of moderately cool weather here, then BAM 90+
mod6: sheet
shinohai: Rarely do we have snow though, usually rains instead
mod6: huh.
mod6: that rain that turns to ice is the worst.
shinohai: We *sometimes* have freezing rain here, feels like pins hitting one in the face
mod6: yeah :/
mod6: sucks
mod6: missouri is famous for the freezing rain
mod6: creats havoc with the powerlines
mod6: well, that was tricky.
mod6: so I'm back up and running with a new empty wallet.
mod6: but i not only needed to back up my wallet.dat, but it initially died because I didn't back up and get rid of the "__db.*" files too.
mod6: so once I moved those out there, then it was happy to start and create a new wallet for me.
mod6: (second test underway with the middle spend step involved)
shinohai: \o/
mod6: hey, check this out:
mod6: % ./db_test.pl mod6
mod6: user_id user_fname user_lname user_nick user_pgp_fp is_project_
mod6: 1 Shane Kinney mod6 027A8D7C0FB8A16643720F40721705A8B71EADAF 1
mod6: shit, bad copy, last field should be 'is_project_owner'
mod6: anyway, this t database is well underway. starting to automate some of this stuff so it can be integrated with tb0t.
shinohai: :D
shinohai: nice
mod6: alright.
mod6: so, got behind a bit on the blockchain, needed to catch up
mod6: ip-172-31-12-97 bin # LC_ALL=C ./bitcoind -datadir=/mnt/btc-dev/.bitcoin listunspent
mod6: [
mod6: {
mod6: "txid" : "a2dd242422a209b26261b5bbb3337c28fb93c013461f8d5ebeae58ca20768a25",
mod6: "vout" : 0,
mod6: "address" : "17cjkFzsLqqJxug6XfkG9YyQ7ctn5kAckZ",
mod6: "scriptPubKey" : "76a9144892feec53df27e9eb7cc1edc882525e9b1cbb2e88ac",
mod6: "amount" : 0.00050000,
mod6: "confirmations" : 31
mod6: }
mod6: ]
mod6: ip-172-31-12-97 bin # LC_ALL=C ./bitcoind -datadir=/mnt/btc-dev/.bitcoin getbalance
mod6: 0.00050000
mod6: (19:57) <@mod6> STEP 1: Send 0.0005 to wallet A
mod6: (19:58) <@mod6> STEP 2: Wait 6 blocks
mod6: (19:58) <@mod6> STEP 3: Send 0.00025 to some other place (no longer needed)
mod6: (19:58) <@mod6> STEP 4: Wait 6 blocks
mod6: (19:58) <@mod6> STEP 5: Send 0.0001 to wallet A.
mod6: (19:58) <@mod6> STEP 6: Wait 6 blocks
mod6: (19:59) <@mod6> STEP 7: Import privkey of pubkey address on Wallet A into Wallet B.
mod6: now doing step 3
mod6: arg!
mod6: error: {"code":-4,"message":"Error: This transaction requires a transaction fee of at least 0.0005 because of its amount, complexity, or use of recently received funds "}
mod6: i'll just send a bit more to that addy and wait a bit more
shinohai: I remember prb had some goofy code that made you wait x confirms before spending ... must be that
mod6: yeah, i've seen it myself in there too.
mod6: wb
asciilifeform: ty mod6
mod6: you missed a few things, but log looks upto date if you care
mod6: logs.bvulpes.com/trilema-mod6
asciilifeform: where's the log again
asciilifeform: ah ty
mod6: step three is done.
mod6: doing step four
mod6: a note on what actually took place because of that error mentioned above: Sent 0.0005 to wallet A
mod6: Sent 0.002 to wallet A.
mod6: Balance was then: 0.00025
mod6: For step 3, sent 0.00013 + 0.0005 (fee) to Some Address 1ABCXYZ
mod6: Remaining balance is 0.00187
mod6: for step 5, I'll send another 0.0001 to wallet A to make the balance: 0.00197
mod6: after that, i'll continue with dumping the key and then importing that pair into a fresh wallet with a scan that should only go back as far as the send to wallet A in step 5.
mod6: ok, well...
mod6: as far as the balance I'm talking about above, that's correct.
mod6: however, wasn't thinking about the change address from that one transaction, so what I'v really got is:
mod6: # LC_ALL=C ./bitcoind -datadir=/mnt/btc-dev/.bitcoin listunspent
mod6: [
mod6: {
mod6: "txid" : "68df09b9e20356c0d86fc28ed7eda05969ce5978988f243f6b739a077ea5676a",
mod6: "vout" : 0,
mod6: "address" : "1GbFMZ7NdZkArDrHfnoy7nE2kx27mF7LMA",
mod6: "scriptPubKey" : "76a914ab04030adb9a6c481a05d6bffda0bfab1ac67a1d88ac",
mod6: "amount" : 0.00137000,
mod6: "confirmations" : 100
mod6: },
mod6: {
mod6: "txid" : "a2dd242422a209b26261b5bbb3337c28fb93c013461f8d5ebeae58ca20768a25",
mod6: "vout" : 0,
mod6: "address" : "17cjkFzsLqqJxug6XfkG9YyQ7ctn5kAckZ",
mod6: "scriptPubKey" : "76a9144892feec53df27e9eb7cc1edc882525e9b1cbb2e88ac",
mod6: "amount" : 0.00050000,
mod6: "confirmations" : 147
mod6: }
mod6: ]
mod6: so I'm gonna go ahead and do step 5, sending another 0.0001 to 17cjkFzsLqqJxug6XfkG9YyQ7ctn5kAckZ
mod6: and then we'll go from there and see.
shinohai: afternoon mod6, still at the rawtx grindstone I see
mod6: yeah, just testing that edge case for beginHeight.
mod6: it's kinda obnoxious without two fully sync'd nodes.
mod6: this certainly could be a lot easier.
mod6: ~somewhat~ easier anyway.
mod6: ok, so, i guess we'll not get quite out of this test that we wanted to get... but we'll get something never the less.
mod6: lol, i've come to far...
mod6: *too
mod6: ok stopping the server, will import 17cjkFzsLqqJxug6XfkG9YyQ7ctn5kAckZ into a fresh new wallet B.
mod6: if I scan back to the first conf of the last tx, we should end up with 0.0001
mod6: which would be correct in again, proving this edgecase.
mod6: post that, will do a rescan and hopefully then we end up with 0.0006
mod6: # time LC_ALL=C ./bitcoind -datadir=/mnt/btc-dev/.bitcoin importprivkey 5678123465... 444025
mod6: real 0m0.661s
mod6: user 0m0.001s
mod6: sys 0m0.002s
mod6: # LC_ALL=C ./bitcoind -datadir=/mnt/btc-dev/.bitcoin getbalance
mod6: 0.00010000
mod6: okie dokie.
mod6: as expected, will start a rescan here in just a few.
mod6: beginning rescan...
shinohai: >.>
mod6: # LC_ALL=C ./bitcoind -datadir=/mnt/btc-dev/.bitcoin getbalance
mod6: 0.00060000
mod6: rescan complete. looks like what we expected.
mod6: so all in all, i'd say that we've replicated the edge case, and it exists, and that there is a work around.
mod6: what's left? I'd say I'd like to see more testing for sure on this.
mod6: And meanwhile we do that testing, I want to look at maybe a bit of a better way to handle those parameters for this function.
mod6: I'll take a look at that too.
mod6: np sir.
mod6: we got time. :]
mod6: oh there you go
trinque: sweet!
mod6: you should be good now. hopefull.
mod6: *hopefully
trinque: yep!
mod6: werd :]
trinque: haxed the makefiles to do it, by using that BUILDER var and adding a Makefile.openbsd
trinque: I've been pondering over the best way to have multi-OS/arch support, would like feedback on an idea I may eventually write up on the blog
trinque: suppose that instead of the codebase accumulating preprocessor hacks to support various platforms, there are instead multiple flows of the V patch graph
trinque: if I were to introduce a vpatch right now for openbsd support, I would *not* expect the next person to patch atop it if he were writing code for linux
trinque: dev for the linux platform would continue on using the same parent I used
trinque: then, as patches *are meaningful* to the openbsd branch, they are reground atop my openbsd patch
trinque: this seems to put an appropriate human step between someone patching the linux version and that new code making it to the openbsd port
trinque: or any other platform
trinque: it also gets really specific about what someone is signing, where it runs, what was meant by the sig
ben_vulpes: huh neat
mod6: wow, cool.
mod6: yeah, the twain shall never meet really. but it's intersting that there could be a way forward.
mod6: were you able to use buildroot easily?
mod6: or did you have to build in linuxcompat into obsd?
trinque: http://btcbase.org/log/2016-12-01#1575551 <<
trinque: I just used the system gcc
mod6: aha, ok
trinque: since openbsd is released as a whole system, I'd say if one wanted to freeze like buildroot, would just pick a release to marry.
mod6: sure.
mod6: sounds good man, nice work.
trinque: oh and where are teh logz?
asciilifeform: trinque: vtronic multiarch is exactly the right thing.
mod6: trinque: logs.bvulpes.com/trilema-mod6
mod6: ben_vulpes:
mod6: ThreadRPCServer method=setgenerate
mod6: 1 processors
mod6: Starting 1 BitcoinMiner threads
mod6: BitcoinMiner started
mod6: keypool reserve 64
mod6: ThreadRPCServer method=getgenerate
mod6: seems to be working
mod6: # uptime 15:57:00 up 336 days, 17:24, 6 users, load average: 1.08, 0.82, 0.42
mod6: for completeness:
mod6: ip-172-31-12-97 bin # LC_ALL=C ./bitcoind -datadir=/mnt/btc-dev/.bitcoin setgenerate false
mod6: ip-172-31-12-97 bin # uptime 16:17:31 up 336 days, 17:45, 6 users, load average: 0.04, 0.22, 0.46
mod6: so yeah, i was able to start it up, see that it was eating CPU, and shut it down on main-net.
mod6: fwiw
ben_vulpes: mod6: let me know if you get it producing blocks off mainnet
mod6: lol, i'll be taking the money and running.
mod6: i did shut the miner down.
ben_vulpes: no i mean outside of mainnet
mod6: yeah, never tried what you're doing.
mod6: i just up and flipped on the miner while on main-net.
mod6: "main-net" being, the defacto-live-bitcoin network.
mod6: it "does stuff" takes CPU to 100% on a single core.
mod6: then shut it down.
mod6: i thought that's what this was all about. i totally missed what you were trying to achieve.
mod6: you're doing a good thing. don't let my dumb test get in the way. i thought the problem was "we can't generate coins!!1", i didn't realize you were trying to do this in another universe.
ben_vulpes: scriptable transaction tests
ben_vulpes: it physically pains me to watch you test this importprivkey patch by hand
ben_vulpes: and for shinohai to be blocked on a full sync
mod6: thanks, but ya it is what it is.
ben_vulpes: mod6: should i expect v.pl to barf when pressing mod6_privkey_tools_wBeginHeight.vpatch without a corresponding signature in .seals ?
ben_vulpes: also interesting is that ./v.pl f shows `mod6_privkey_tools_wBeginHeight.vpatch' as coming /before/ makefiles.vpatch
ben_vulpes: and pressing makefiles.vpatch and then mod6_privkey_tools_wBeginHeight.vpatch blows up in the way i'd expect it to
ben_vulpes: eg complaining about hash mismatches
shinohai: O.o
mod6: <+ben_vulpes> mod6: should i expect v.pl to barf when pressing mod6_privkey_tools_wBeginHeight.vpatch without a corresponding signature in .seals ? << not because of that, no.
mod6: <+ben_vulpes> also interesting is that ./v.pl f shows `mod6_privkey_tools_wBeginHeight.vpatch' as coming /before/ makefiles.vpatch << not actually very interesting
mod6: they are both leafs, and in the flow, it's hard to represent that visually. just press all the way through `makefiles.vpatch'
shinohai: Mine always showed up `before` makefiles in flow but pressed/built fine
mod6: <+ben_vulpes> eg complaining about hash mismatches << if your thing is puking, i suspect that you need to clean out your patches/.seals and start again.
mod6: maybe you have an old vpatch?
mod6: can you paste me an error?
mod6: or where the press failed?
mod6: as a matter of course, you shouldn't ever fail when pressing a WILD vpatch.
mod6: well, at least not for not having a corresponding .seal.
mod6: we ~should~ fail, if a .seal doesn't verifiy with the corresponding vpatch -- but that happens much before "press time".
mod6: well,... at least code-wise.
mod6: but anyway, yeah, shoot me an error, and we'll get you figured out
ben_vulpes: well hang on, i thought a patch without a signature shouldn't press without passing the press command some form of WILD=true flag
ben_vulpes: anyways, on to the next point, that import privkey patch doesn't apply on top of the makefiles patch
ben_vulpes: i can regrind it so that it does, that's not a big deal
ben_vulpes: but my observed behavior is that pressing to the import privkey patch does not result in the makefiles patch being applied first
ben_vulpes: and after i press to makefiles, then pressing to privkey_tools fails because of hash mismatches. which makes sense, because it doesn't appear to descend from makefiles.
ben_vulpes: and this is all in a freshly synced work directory
ben_vulpes: into the patches directory of which i manually emplaced the privkey_tools vpatch
asciilifeform: ben_vulpes: if at any point the correct operation of a vtron is in doubt, you can check with bare hands
asciilifeform: simply apply the patches in the expected order, and hash the result
ben_vulpes: asciilifeform: is it correct for a v implementation to press a patch without a corresponding signature without passing a wild flag to the press command?
asciilifeform: never.
asciilifeform: (i can't speak for ben_vulpes's or mod6, but mine certainly would not.)
ben_vulpes: i never implemented wild patch handling in mine. all patches needed signatures.
asciilifeform: imho this is mistake
ben_vulpes: aye
asciilifeform: one ought not to sign items not intended for distribution
ben_vulpes: 'twas an incomplete implementation.
asciilifeform: (outside of a few rare special cases, e.g., launch codes)
ben_vulpes: and yes, the change would've been a bare few lines. nevertheless. the correct behavior is for a press operation to fail without a signature for each patch, unless v is expressly instructed to press wild patches.
asciilifeform: noshit.
asciilifeform: if a vtron presses an unsigned patch without being asked very politely and specifically, then it is about as useful as a parachute that doesn't slow your fall to the ground
asciilifeform: i.e. entirely useless regardless of what other merits.
mod6: <+asciilifeform> (i can't speak for ben_vulpes's or mod6, but mine certainly would not.) << yes, mine does.
asciilifeform: waiwut?! it presses 'wilds' by default?!
mod6: ofc.
mod6: anyway, i can add a feature for that.
mod6: but its meant that if you don't want a wild patch in your flow, don't put one in there.
mod6: and while you're at it, check your flow, before you press.
asciilifeform: mod6: 'don't put one in' is entirely wrong.
asciilifeform: a correct vtron lets you turn the knobs ~solely~ by altering your wot.
asciilifeform: e.g., if i want an official trb foundation release, i set my wot to ' mod6 , ben_vulpes '
asciilifeform: if i want to build a new node -- i set it to ' asciilifeform '
asciilifeform: etc
mod6: eh, well, maybe my thing doesn't work the way its supposed to then
asciilifeform: to be fair, my original vtron also was not ideal, it never checked result hashes for instance.
asciilifeform: (nor had any way of nonlinearly climbing the tree)
mod6: would you consider then, writing a Vtron that we can all use?
asciilifeform: mod6: iirc you have the best one currently.
mod6: Like would you consider making your tool general use?
mod6: It sounds like, you're saying, that mine sucks shit.
asciilifeform: mod6's is much closer to 'general use' than mine !
mod6: because it doesn't do what it is supposed to do.
asciilifeform: it has many fewer bugs than mine!
asciilifeform: afaik -- just this one.
mod6: Would you then consider, doing a black-box audit of mine -- to see what other envivitable shit i've missed/
asciilifeform: i would, but i've forgotten perl more or less entirely.
asciilifeform: (which happens whenever i leave it alone for a year or so)
mod6: [black box]
mod6: don't even look at the perl. just use it.
mod6: see what makes sense.
mod6: or does not.
asciilifeform: i began to do this, recently.
asciilifeform: but notice, ben_vulpes unearthed surprise before i did.
asciilifeform: or recall, the '+++' incident ?
asciilifeform: arguably that is 'bug' in vdiff, even.
asciilifeform: which i still have nfi what to do about.
asciilifeform: i regard vtron as a work in progress. gnudiff, for instance, has no permanent place in the world.
mod6: what in the fucking fuck
mod6: http://therealbitcoin.org/ml/btc-dev/2016-February/000214.html
mod6: A non-text attachment was scrubbed...
mod6: Name: v.pl
mod6: Type: application/x-perl
mod6: Size: 30386 bytes
mod6: Desc: not available
mod6: URL: <http://therealbitcoin.org/ml/btc-dev/attachments/20160220/v-0001.pl?sha1=893186bccbed0661a4e5ea3466a7c118e4624d1b>
mod6: why does his thing now re-write the file names???
mod6: v-0001.pl ?!
asciilifeform: mod6: iirc it used to
mod6: so ... i could add a check so that it'll never press something that is WILD, unless a specific flag is passed.
mod6: but if that's not how it's supposed to work, then how would I ever test anything?
mod6: am I expected to sign everything I test?
asciilifeform: that is exactly how it is supposed to work. and no, you don't have to sign your test drafts, it is exactly what wild mode is for.
asciilifeform: i thought this was clear from my original vtron.
mod6: it's been too long tbh. i haven't looked at your thing in years.
mod6: i'll have to check it out.
asciilifeform: fortunately it is very quick read !
mod6: # Permit the use of patches no one has yet sealed. Use this ONLY for own dev work!
mod6: parser.add_argument('-wild', dest='wild', default=False, action="store_true", help='Permit wild (UNSEALED!) vpatches.')
mod6: ah, ok.
mod6: well, i'll have to throw something together for that
mod6: christ on a cracker
mod6: let me know if you find any other bad shit in there.
mod6: anyway, ben_vulpes this still isn't why youre having issues. let's figure that out when you get a chance.
mod6: this channel no longer serves its intended purpose.
mod6: just revert to #trilema.
shinohai: well
ben_vulpes: i'm still in the dark re intended purpose
mod6: Probably will use this channel for now as a central place to coordinate testing for forthcoming V99994.
asciilifeform: neato mod6
mod6: Will be a number of days yet until I get all the automated tests updated, etc. Will announce when ready.
ben_vulpes: hey mod6 can you help me figure out where v.pl determines the filename to check the hash of post-press?
mod6: sure.
mod6: with v.pl V99995
mod6: there is a subroutine on line 292: press_vpatches
mod6: we call: verify_pressed($press[0], add_pressed($vp)); on line 308 for each vpatch
mod6: the meat of what you're looking for is this line 330 of verify_pressed:
mod6: my $fp = $press_dir . "/" . get_filepath($src_file_name);
mod6: which appends the press dir, to a '/', then the return value from get_filepath, a subroutine located on line 318
mod6: which takes a src_file_name as a parameter which comes in the form of something like a/bitcoin/src/main.cpp, and then gets returned as 'bitcoin/src/main.cpp'.
mod6: am I making sense here?
mod6: (essentially, get_filepath, lopps of a preceeding 'a' or 'b' from the front of what is scooped up from what is actually denoted inside the vpatch itself.)
mod6: then, finally, on line 331 we do this: my $hashed = `sha512sum $fp`;
mod6: where $fp is set above as: my $fp = $press_dir . "/" . get_filepath($src_file_name);
ben_vulpes: mod6: yeah, after phf's commentary i wondered if you'd hit the same caltrop as i did. what does your implementation do if the vpatch is made from directories a/ and b/ ?
mod6: i couldn't make heads or tails out of what he was talkin about.
ben_vulpes: mod6: try this experiment
ben_vulpes: cook a vpatch from directories "alphabet" and "betazoid"
ben_vulpes: vdiff alphabet/ betazoid/
mod6: you can't do that.
ben_vulpes: sez who?
mod6: it needs to be: ./vdiff a b
mod6: well, thats just what it's always been.
ben_vulpes: argument ad raisinum
mod6: wut
ben_vulpes: "hysterical raisins" or "historical reasons"
mod6: anyway, i'll try it just the same and see what it does.
ben_vulpes: i got five bucks sez your hash check fails
mod6: can i use 'alpha' and 'beta' instead of whatever and whatever?
ben_vulpes: sure
ben_vulpes: anything that's not "a" and "b"
ben_vulpes: although for rigor's sake you may as well make it "mercury" and "venus" so's to doubly confuse the function that strips leading "a/" and "b/"
mod6: bah
mod6: ok
ben_vulpes: http://logs.bvulpes.com/trilema-mod6#12 << i mean to say if you cook a vpatch *not* from directories a/ and b/
mod6: http://wotpaste.cascadianhacker.com/pastes/2PQdI/?raw=true
mod6: to fix this, I needed to change line 320 from: $fp =~ /^[a|b]\/(.*)$/;
mod6: to: $fp =~ /^.*\/(.*)$/;
mod6: this makes me nervous, like people may really do some dumb shit...
ben_vulpes: i'm miserably bad at regex-eval-in-head but i think that only handles 1 level of directories, and i suspect we should be able to handle the general case
mod6: you mean that you only wanna deal with the file name itself????
mod6: i'd never do that
ben_vulpes: no, but that there is no guarantee of depth before we get to the source directory
mod6: agree, but that's up to them to define how their directory structure is laid out
mod6: i can maybe accept not searching for [a|b], but whatever the top level... at least, with a metric fuckton of testing, i could be convinced to sign something like that.
mod6: maybe
mod6: anyway, in my mind source directory really isnt a thing.
mod6: it's everything below a|b.
mod6: that's the project.
ben_vulpes: i contend that v must work on patches made from arbitrary directories on the user's hard drive
mod6: i saw phf say something like; /User/someshit/otherplace/downhere/in/this/hole and /home/blah/blah/blah and wanted to diff these two and have it come out right..
mod6: at least that's what I picked up in the .69 ms i thought about that comment.
ben_vulpes: patch tool handles this correctly. v, however, does not.
mod6: which to me, seems like bad form.
ben_vulpes: your and my v implementations, when going to check the hashes, will lose their minds over this.
mod6: there may be a reconsile this without fucking everything up, but it escapes me at the moment.
mod6: like i said, this could be super dangerous.
mod6: and im in zero hurry to change it.
mod6: do you see what I mean though? i can see, maaaaybe if the different directory depths are the same, and you would lop off the same amount of dirs from the top...
mod6: but if they're of different depth?!
mod6: anyway, yeah, i just don't know really.
ben_vulpes: well somehow patch handles this correctly
mod6: yeah but you have to pass it a depth flag.
mod6: i guess id have to look into it.... i'll chalk this up to "nice to have" for the time being.
mod6: but tbh, it's not something I would want in my personal v.
ben_vulpes: lol i want my personal v to know wtf it pressed
mod6: how do you not?
mod6: my v knows what it pressed.
mod6: does yours not do the same thing? and ftr, i don't understand lisp, so when reading yours i have the faintest clue as to what is actually going on.
ben_vulpes: yours does not if i hand it a patch made from directories that weren't a/ and b/
ben_vulpes: mine either
mod6: mine ~does~ know what is pressed... and tells you that someshit went wrong when it can't figure it out.
ben_vulpes: i'm working to upgrade mine so that it knows which precise files on disk it pressed when handed a patch cooked from directories other than a/ and b/
mod6: it ~checks~ the expected output, and if what it finds isn't expected, it halts and tells you.
ben_vulpes: yeah mine errors out if anything goes wrong as well
mod6: this is correct.
ben_vulpes: but this a/ b/ shit is unnecessary magic strings
mod6: maybe.
mod6: i see the probability of massive problems much higher from using any ole dirs as opposed to that though.
ben_vulpes: it may be an accidental and necessary pruning of the possibility space but i'm not convinced yet
mod6: im not convinced either.
ben_vulpes: GREAT LET'S ARGUE MORE
mod6: i can agree that this type of thing should be eliminated where possible, but only when the risk is low. and that might just come from clever programming and exaustive testing.
mod6: but it's high risk.
mod6: at least to me.
mod6: <+ben_vulpes> GREAT LET'S ARGUE MORE << lol.
ben_vulpes: at the very least right now both implementations demand patches be cooked from directories named a/ and b/
ben_vulpes: and at the very least this is an implicit part of the spec that's worth pointing at and asking the peers to opine about
mod6: i just see this as a "nice to have" not a requirement right now. for instance, if everyone insisted on THIS being a MUST CHANGE right now, it would grind lots of stuff to a halt. i see this as wildly dangerous change.
mod6: yeah. i can see how there should maybe be a discussion around it.
mod6: but to me, it's low priority.
mod6: for me, getting the spec of vtronics working properly and vetted is much higher.
mod6: so one trap is the depth .
mod6: so if a guy goes to press, he needs to pass a prune depth to the press flag.
mod6: just like with patch -p1 or whatever.
mod6: and off the top of my head, this would be fine, with the edge case that his directory depth is not equal.
mod6: then we're fucked in the goatass.
mod6: i guess, he'd have to do like...
mod6: well, nevermind that still wouldn't work.
mod6: take these two:
ben_vulpes: well up until today nobody's even mentioned constraints on the directories from which a vpatch can be cooked
mod6: /User/someshit/otherplace/downhere/in/this/hole/foo.txt
mod6: /home/blah/blah/blah/foo.txt
mod6: how to deal with this? even if you want to throw out all the dirs above foo.txt for one its -7 and for the other its -4
mod6: hmm. come to think of it... now i've confused myself.
ben_vulpes: see #trilema, patch does magic when applying patches
mod6: does patch prune from the bottom up or top down>
mod6: i gotta check quick.
ben_vulpes: bottom i believe
mod6: -pnum or --strip=num
mod6: Strip the smallest prefix containing num leading slashes from each file name found in the patch file. A sequence of one or more adjacent slashes is counted as a single
mod6: slash. This controls how file names found in the patch file are treated, in case you keep your files in a different directory than the person who sent out the patch. For
mod6: example, supposing the file name in the patch file was
mod6: /u/howard/src/blurfl/blurfl.c
mod6: setting -p0 gives the entire file name unmodified, -p1 gives
mod6: u/howard/src/blurfl/blurfl.c
mod6: without the leading slash, -p4 gives
mod6: blurfl/blurfl.c
mod6: and not specifying -p at all just gives you blurfl.c. Whatever you end up with is looked for either in the current directory, or the directory specified by the -d option.
mod6: so it is from top to bottom.
mod6: if we were to mirror this, i think we'd be screwed.
mod6: at least, from the perspective of that edge case.
mod6: we'd have to do the reverse to make this something that would be correct.
mod6: but this ~is~ a can of worms no matter what.
ben_vulpes: we might be using bottom and top backwards from each other
mod6: oh, maybe, i'm saying / as root, meaning the leftmost, meaning the 'top', and the file at the leaf as the bottom.
mod6: and the reverse being user passes -p0 and would simply get 'foo.txt'
ben_vulpes: yeah, we're saying the same thing but mirrored
mod6: what happens if the guy says, -p1 with the reverse? he gets nonsense, as will attempt to press: hole/foo.txt and blah/foo.txt
mod6: i am open to suggestions on how to do this properly and safely.
davout: i'm immensely sorry to bother with my incredibly noobish questions, but is the flow "get V from foundation wwwtron; ./v init http://thebitcoin.foundation; ./v.pl p v trb54 makefiles.vpatch" supposed to work properly ?
davout: oh wait a fucking second
mod6: yes, it should, but you're missing the .pl part of the first command.
davout: yeah yeah, typing from memory here :)
mod6: what problem are you having? can you provide a log?
mod6: where are you seeing these instructions?
davout: the problem seems to be that /me is an idiot and confused the OSX shell tab with the proper linux box shell tab and tried to press on stupid macos
mod6: oh, ok.
davout: and since osx lacks a proper sha512sum pressing can't possibly work
mod6: well, it certainly doesn't know ...
davout: i found the instructions in v's help messages
mod6: yeah exactly, it doesn't know about `shasum -a 512` only `sha512sum` on a linux box
mod6: ok good deal :]
ben_vulpes: welcome back davout
ben_vulpes: we missed you bb
ben_vulpes: never leave mauritius
davout: ben_vulpes: what are you talking about! i'm intermittently present around here
ben_vulpes: davout: your two-week running linecount average is back in the noticable range
davout: i'm an ideas man yo
ben_vulpes: we all have our gifts
davout: trb currently building, fingers crossed!
ben_vulpes: dat make ONLINE=1...
mod6: you'll need GCC version 4.x
ben_vulpes: davout: not much luck to it, if it blows up at this point the causes are always interesting
davout: ben_vulpes: found out about it *after* fetching the dependencies manually
davout: mod6: apparently I have 4.8.4 \o/
mod6: should be fine.
davout: out of curiosity, what's your hacking-on-trb workflow?
davout: for example, how do oyu figure which files you touched after an hour of hacking around?
mod6: i don't really work like that.
davout: with git it's pretty trivial to figure it out, but without it...
mod6: i mostly read, and then read some more.
davout: make a perfect mental image of the necessary patch and then write it in one go? :D
ben_vulpes: lol mod6 the code stoic
mod6: then i create an 'a', and 'b', directories with the same versions, exactly, the same, in both.
mod6: as I touch files, i do so in 'b'.
mod6: and as I go, if I want to see what I've done, i just vdiff a b
mod6: typically that's how i go about it. infact, there even is some redundancy. but its meticulous. very much unlike my spelling :]
ben_vulpes: i have a trb source directory under git control, compilation via emacs that leans on buildroot
mod6: sometimes when I want to build, i rebuild the entire orchastra. this is usually when i have a new vpatch or something similar.
ben_vulpes: i ~never rebuild the entire orchestra, except when building a new release from scratch.
mod6: but if i've already pressed and built the orchastra, and want to add a debug statement (for instance), i'll rebuild just the bitcoind binary by itself.
mod6: i rebuild entire thing a lot. just to ensure, end to end, that everything is correct and working.
mod6: it might be overkill lot of times, but i like to be sure.
ben_vulpes: i do not subscribe to this 'just in case' recompilation of the entire linux kernel process
mod6: you ever done QA for a living?
ben_vulpes: no, and hope to never :D
mod6: heheh.
davout: lol
mod6: every developer should have to start in QA.
ben_vulpes: dog if something changed such that the linux kernel i have on disk broke
davout: doing QA you get to realize that: most users are idiots, most developers are also idiots
ben_vulpes: given that i NEVER TOUCHED THOSE FILES EVER FUCKING EVER
mod6: i don't want to make it sound like i do this like, 69 times a day. i just really want to ensure that it presses cleanly with a new vpatch.
mod6: then it builds correctly from there.
mod6: sometimes i just rebuild the binary itself to check something or whatever.
mod6: but yeah, super meticulous process for me.
mod6: at least, i try very hard to be that.
mod6: a lot rides on these things, so, caution, slow moving and read, read, read are virtues.
ben_vulpes: well read read read for sure, but recompile the linux kernel recompile the linux kernel recompile the linux kernel is a level of paranoia i aspire to
mod6: haha.
mod6: it's actually a good thing in a way.
mod6: because between shinohai and i, we compile this thing on a lot of different linux environments. 'tis how we discovered the fuckin ncurses thing.
mod6: speaking of which...
mod6: it'd be really cool some day to get like a virtualization environment where we can spin up like a bunch of different versions of ubuntu, gentoo, deb, others? and then have bamboo or something kick off a full orchastra build on all of 'em.
mod6: we're basically doing this now, but it's all "manual"
mod6: soooo much time is spent doing QA. which is totally 100% necessary. but it'd be great to automate some of this labor.
mod6: i even have aspirations to write 10,001 unit tests for trb.
mod6: and then further, functional tests via cucumber or something similar.
mod6: having these tests in place will give us the confidence we need to make some of the intricate changes that are wanted on the horizon.
mod6: i've just started scratching the surface on cucumber tests for trb, but obviously a lot more are needed.
mod6: someday, i can see us having a group of automaters just to do exactly that; write new tests, maintain the existing tests.
davout: ok, so it built! mazeltov to the tmsr QA dept.
mod6: i actually hoped, about a year ago to be further along with this, but alas, i only have so many hands.
mod6: davout: nice!
mod6: good work.
davout: lol, what 'work'?
mod6: just getting v setup, all the things in place, and building it.
davout: now i need to plug a trusted node to it
mod6: there ya go.
davout: doesn't alf have a 'public service' trb node ?
mod6: do take note of suggested startup string in this howto: http://thebitcoin.foundation/trb-howto.html
mod6: http://thebitcoin.foundation/trusted-nodes.html
davout: ah ty
mod6: yw.
ben_vulpes: davout: 'coracle' runs at 216.151.13.77
davout: ben_vulpes: yours?
ben_vulpes: davout: aye
davout: mod6: for some weird reason, nohup seems to have no effect, as soon as i close my shell, bitcoind exits (cleanly)
davout: i guess i could screen the whole thing, but i'm wondering if there's something obvious I'm missing here
davout: starting it with -daemon did the trick, still curious about what caused nohup to apparently not work
mod6: weird.
mod6: yeah not sure.
ben_vulpes: davout: how are you using nohup?
davout: LC_ALL=C nohup ./bitcoind -myip=<yourip> -addnode=trusted_node_1 -addnode=trusted_node_2 -verifyall 2>&1 &
ben_vulpes: but yeah, daemon is the trick. fwiw davout i use runit to monitor process uptime and restart it when/if it dies
davout: as per thebitcoin.foundation kwikstart guide
ben_vulpes: cool, thanks for the nohup pointer
davout: ben_vulpes: please tell that setup is managed by god inside a docker container on aws
ben_vulpes: davout: neg
davout: did much more come out of http://btcbase.org/log/2015-01-02#962841 ?
ben_vulpes: bare metal in a dc i can walk into, runsvdir and bitcoind
davout: because that's the main reason i keep a node in the first place other than for motherland
ben_vulpes: davout: mod6 is cooking createrawtx, or was before recent vtronix kerfuffle
davout: such paleo-diet of computing
ben_vulpes: it's not a lifestyle davout
ben_vulpes: IT'S WHO I AM NOW
davout: what i usually use is a mix of gettxout, getrawtransaction, sendrawtransaction, signrawtransaction, createrawtransaction, listunspent
davout: not sure what the best approach is to help get those, or a subset of those, in
davout: or out actually, i'm far from convinced these belong in bitcoind itself
ben_vulpes: davout: you read mircea_popescu's "how to cut the wallet"?
davout: i remember something like this, plox to link?
ben_vulpes: davout: trilema.com/2016/how-to-cut-the-wallet/
davout: ty
asciilifeform: <davout> for example, how do oyu figure which files you touched after an hour of hacking around? << is why i made the 'origin' command in my vtron
asciilifeform: <davout> doesn't alf have a 'public service' trb node ? << not 1 but 2
davout: :)
trinque: also deedbot.org runs a node
trinque: sometimes it's even up to date
asciilifeform: trinque: iirc mircea_popescu also has a few.
asciilifeform: and bunch of other folks.

2016-11-30 | 2017-1-1