[00:17] creationix: isaacs: I volunteered to write a compiler in coffee-script, we'll see how far I get [00:17] isaacs: creationix: neato! [00:17] isaacs: tail-eating languages are always the coolest. [00:18] orlandov: <3 self hosting projects [00:18] orlandov: i think that's so cool [00:18] creationix: I actually just got hired on with jashkenas yesterday, I'm really excited about the stuff we're going to release later this year [00:18] isaacs: hired where exactly? [00:18] orlandov: congrats :) [00:18] isaacs: can you say? [00:18] creationix: DocumentCloud [00:18] creationix: I got on as a JS developer [00:19] creationix: I honestly don't know he write so much good code in such a short time [00:19] creationix: I thought I was a prolific coder [00:20] creationix: *know how he writes... [00:20] creationix: if anyone has experience writing parsers in JS that would help a ton, the rest of the compiler is pretty straightforward [00:21] orlandov: creationix: i believe inimino was building a JS parser [00:21] orlandov: http://inimino.org/~inimino/blog/peg_first_release [00:21] sudoer has joined the channel [00:21] creationix: hmm, that might be useful [00:22] creationix: I think the current compiler uses racc [00:22] orlandov: now i wish someone would port jemplate builder from perl to js [00:25] jed has joined the channel [00:28] creationix: CoffeePot - A CoffeeScript compiler written in CoffeeScript http://bit.ly/5QDwGV [00:35] isaacs: it's a bit odd that variable? is false for null. [00:35] isaacs: undefined !== null [00:41] jacobolus has joined the channel [00:42] orlandov: that's gotta be a bug? [00:43] jacobolus has joined the channel [00:46] hassox has joined the channel [00:53] olivvv has joined the channel [00:58] elliottcable: creationix: ain’t that a bit, uh, recursive? [00:59] elliottcable: creationix: shouldn’t you write a coffeescript compiler in *JavaScript*, so you can compile in the browser? [01:01] inimino: elliottcable: once the compiler compiles itself, you have a compiler in JavaScript [01:01] isaacs: elliottcable: coffeescript can compile to javascript. if you can do it in js, you can do it in cs [01:01] inimino: elliottcable: you just didn't have to write it [01:01] elliottcable: isaacs: yes, but you still have a dependency on the Ruby CoffeeScript, then [01:01] elliottcable: go paw [01:01] elliottcable: oops [01:01] isaacs: you compile it the first time using the ruby kit, and now you have a js compiler, and you can use it to compile the cs compiler, and test that the new output equals itself. [01:01] elliottcable: >,> [01:02] elliottcable: isaacs: yes, but my point still stands: recursive dependency [01:02] isaacs: just like how the first c++ compiler was in c with some fancy macros, and pretty much every c++ compiler since has been in c++ [01:02] inimino: not once you've compiled the compiler [01:02] elliottcable: even if you have a way out, with the Ruby compiler d-: [01:02] isaacs: or how the first c compiler was in asm, but ever since, they've been in c [01:02] isaacs: or narcissus in javascript [01:03] inimino: just like how GHC is written in Haskell... and uses features that only GHC has [01:03] inimino: or gcc, for that matter [01:04] isaacs: inimino: right, once you've got the thing, it can iterate on itself by adding features which can be specified in terms of other existing features. [01:04] inimino: yep [01:04] isaacs: so version 2 of the compiler can do things that weren't in the language for version 1 of the compiler, even though it's been written in the language since version 0.2 [01:04] isaacs: computer science warms my cockles. [01:05] inimino: ACTION has a couple projects operating on this principle [01:06] creationix: yep, exactely [01:07] isaacs: i was thinking it'd be a fun exercise to write a lambda calculus compiler that compiles to javascript. [01:07] isaacs: not fun as in "useful", mind you. [01:08] isaacs: and i'd replace λ with ƒ since option-F is easier to type [01:08] creationix: isaacs: that does sound like fun [01:09] isaacs: in principle, you can always do λx.y --> function (x) { return y } [01:10] isaacs: and λx.y(z) --> (function (x) { return y })(z) [01:10] creationix: My goal with coffeepot is to be able to distribute it as a single html file with embedded js, css, and data uri's for graphics [01:10] creationix: and of course a node library [01:11] isaacs: nice [01:11] jed: creationix: wow, that would be incredible. [01:11] elliottcable: creationix: why? [01:11] elliottcable: creationix: I must be missing something, what’s the point of distributing as HTML? [01:12] CIA-78: node: 03Ryan Dahl 07master * r1b9eaf4 10/ bin/node-waf : Update node-waf version to 1.5.10 - to match tools/wafadmin - http://bit.ly/5qrWXq [01:13] creationix: for fun [01:13] elliottcable: yesyesyes but I mean [01:13] elliottcable: what use is it if you have to yank it out of an HTML file before using it [01:13] elliottcable: o_O [01:13] creationix: of course that won't be the only way I distribute it [01:13] elliottcable: I didn’t think so; I’m saying, what’s the point to distributing it as HTML at all? [01:14] elliottcable: or am I missing something (probably)? [01:14] creationix: something like this [01:14] creationix: http://tryruby.org/ [01:14] creationix: but in a local static html file [01:14] creationix: for learning and playing [01:14] creationix: repl in the browser [01:14] elliottcable: Oh. You’re talking about an interactive CoffeeScript REPL in an HTML file? [01:14] elliottcable: yah, I getcha [01:14] elliottcable: Was planning to do the same thing with Paws [01:15] elliottcable: my bad, I just somehow misread ‘distribute as an HTML file’ as ‘use HTML as a replacement for tar’ [01:15] elliottcable: which seemed… WTF. [01:15] elliottcable: (-: [01:15] creationix: lol, agreed [01:15] elliottcable: creationix: well, you may be interested in what I’m writing *at this particular moment*, then [01:16] elliottcable: creationix: it’s a pretty printer that’s intended to work both in the terminal and the browser, equally well [01:16] elliottcable: creationix: it’s eventually going to power the Paws terminal/browser REPL [01:16] elliottcable: creationix: but it’s written in JS, not Paws [01:16] creationix: that would be nice [01:16] elliottcable: creationix: http://github.com/elliottcable/Percival [01:17] elliottcable: atm, I’m trying to get it to intelligently handle ANSI escape codes /-: http://cld.ly/b6ziv [01:17] dnolen has joined the channel [01:22] creationix: very nice [01:22] elliottcable: I love it when bugs fix themselves :D [01:25] isaacs: elliottcable: is that an actual-size screenshot? [01:26] elliottcable: isaacs: uh? [01:26] isaacs: http://www.cld.ly/b6ziv [01:26] elliottcable: isaacs: what do you mean? [01:26] isaacs: i mean, is that the actual pixel size of the font you use? [01:26] elliottcable: … at the moment, yes? [01:26] elliottcable: it changes fairly often, depending on whether I’m feeling productive or not [01:26] elliottcable: more productive == it gets smaller [01:27] inimino: isaacs: that would be pretty useful, and fun [01:27] isaacs: elliottcable: holy hell, you must have some very good vision. make sure to go outside and get lots of sun and eat your carrots. [01:28] isaacs: ACTION is deeply nearsighted, uses at least 16pt for everything [01:29] creationix: "command +" is your friend [01:29] creationix: or mine at least ;) [01:29] isaacs: srsly!! [01:29] inimino: indeed [01:29] r11t has joined the channel [01:29] isaacs: because of the wicked astigmatism, I also have problems with high-contrast. reability script is quite winful: http://lab.arc90.com/experiments/readability/ [01:32] richtaur: whoa that is lovely [01:33] creationix: perfect, I lost my glasses a couple of weeks ago. Not nearsightes, but pretty bad astigmatism [01:33] creationix: (hence all the typos) [01:37] alexiskander has joined the channel [01:39] unom1 has joined the channel [01:41] creationix has left the channel [01:49] charlenopires has joined the channel [01:50] lattice1: gah. git is making my brian bleed. maybe someone here can help. ;-) [01:53] lattice: I have a branch off of isaac's branch that I've added mx/srv/txt support to the dns module, and I'd like to rebase to ry's master to make it possible to apply the changes. [01:53] lattice: my config looks like: [01:53] lattice: [remote "origin"] [01:53] lattice: url = git@github.com:blaine/node.git [01:53] lattice: fetch = +refs/heads/*:refs/remotes/origin/* [01:54] lattice: and I'm on a branch called "dns-rrtype-support" [01:54] bryanl has joined the channel [01:55] lattice: issuing the command "git pull —rebase git://github.com/ry/node.git master" seems to bring me into some kind of special conflict hell. :-/ [01:56] bryanl has joined the channel [01:58] lattice: oh. Umm, yeah, nevermind. [01:59] elliottcable: isaacs, creationix, and inimino: I have some sort of ridiculous vision, 20-6 or something like that [02:00] elliottcable: isaacs, creationix, and inimino: I keept waiting for my 24/7 sleepless eyeglassesless computer use to ruin them… but, oddly enough, they’ve gotten *better* in the last few months (I’ve barely left the computer for about 3 years, minus a few completely computer-less vacations) [02:01] binary42 has joined the channel [02:03] alexiskander has joined the channel [02:10] scudco has joined the channel [02:37] eddanger has joined the channel [02:38] lattice: ryah: thanks - the ipv6 test is working for me; that result suggests that 'dig' is able to resolve IPv6 addresses, but for some reason udns isn't. [02:44] ryah: lattice: are you blaine? [02:45] lattice: ryah: yes, sorry. :-) [02:45] micheil has left the channel [02:46] ryah: lattice: yeah, i'm not sure why it's not passing here... don't have time to look at it right now [02:46] lattice: no worries - would you rather I kept it in, or disable the ipv6 tests for the moment? [02:47] ryah: i'd rather find out why it's not working :) [02:52] lattice: ryah: hehe. Definitely; that code hasn't changed, so it's probably a pre-existing problem that wasn't surfacing since the DNS tests were disabled. [02:53] lattice: I'll push with the test enabled, and let the list / you look at it. I can't reproduce it here, unfortunately, and it's way past my bedtime anyhow. ;-) [02:53] ryah: okay [02:54] eddanger has joined the channel [02:58] micheil has joined the channel [02:59] micheil has left the channel [02:59] jed has joined the channel [03:21] isaacs: ryah: how awful is it to do something like createChildProcess("node", ["some-script.js"], process.ENV) and then IPC to/from it? is there some smarter way to do this already? [03:22] isaacs: i'm tossing around ideas for having a supervisor-type of program that monitors a web server, restarts it when it crashes, etc. [03:25] ryah: isaacs: i think it's okay. felixge is doing it. [03:25] isaacs: well, if felix is doing it, then it must be ok, because he's way smarter than i am. [03:25] isaacs: thanks! [03:30] ryah: isaacs: BDFI? [03:31] isaacs: bdfl = benevolent dictator for life. [03:31] isaacs: ie, the guy who owns the project, cares the most about it, and makes arbitrary preference-based decisions that the community would otherwise argue about forever. [03:32] ryah: what do you think about blaine's patches? [03:32] ryah: for url? [03:32] isaacs: well, it's mostly renaming for the sake of renaming. [03:33] isaacs: imo, following the browser's naming convention for javascript things is the Right Thing, but i'm a DHTML javascripter first, and blaine is a server-side rfc afficianado, and he and i will never agree on that. [03:33] isaacs: imo, that aspect of it is your call. [03:34] isaacs: for the other things that he changed, it's hard to separate from the naming changes. if you say "we're gonna call our urls by the same names that python uses", then great, i'll pull his name changes, and it'll be easier to evaluate the other (relatively minor) parts of the patches. [03:34] ryah: less patching = less work :) [03:34] isaacs: otherwise there's a lot of combing through to see what he's doing, and how it differs if you DIDNT rename everything first. [03:34] ryah: ACTION waits for more complaints [03:35] isaacs: but, i think this is a symptom of a deeper issue. [03:35] isaacs: namely, there's no single "this is node's philosophy" document to point to to resolve stuff like this. [03:36] hassox: "be awesome to each other" ? [03:36] hassox: that's a good philosophy [03:36] isaacs: haha [03:37] isaacs: ryah: it'd help you get more contributions, as well, because people would have a better guess as to whether or not their work will gain acceptance, or if they should instead pursue building their idea as a standalone project or something. [03:38] hassox: ryah: would it be true that if I was to say that it should never block to wait? [03:38] hassox: that cpu usage should be maximised [03:38] hassox: ? [03:39] ryah: sorry - i have to go [03:39] isaacs: np [03:39] isaacs: give it some thought. i'm willing to help. [03:39] ryah: hassox: good programs should maximize cpu when hit with very high loads [03:39] ryah: hassox: servers [03:39] ryah: not sure if that's what you're asking [03:39] hassox: yeah [03:40] hassox: that's basically it [03:40] hassox: I'm looking for some nice quotes ;) [03:57] brosner has joined the channel [03:59] RayMorgan has joined the channel [04:07] micheil has joined the channel [04:07] micheil has left the channel [04:18] grantmichaels has joined the channel [04:30] kriszyp: is the url module new? is it supposed to be in 0.1.24? [04:30] martint-t has joined the channel [04:31] jed: kriszyp: it's only in master, i think. [04:32] kriszyp: ok, thanks [04:32] jed: kriszyp: apparently not in http://github.com/ry/node/tree/v0.1.24/lib/ [04:32] kriszyp: so I guess I shouldn't upgrade quite yet... [04:32] jed: master is stable for me, but i guess it depends on your needs. [04:32] isaacs: kriszyp: you should just live on teh edge, man [04:33] kriszyp: yes, you are right, i should [04:33] isaacs: actually, ryah_away's master is typically quite stable. [04:34] kriszyp: can I download and install it from git just like from the download link? [04:34] kriszyp: or is there an extra build step? [04:34] isaacs: kriszyp: just replace "tree" with "tarball [04:34] isaacs: http://github.com/ry/node/tarball/master [04:34] isaacs: then make like we all know and love [04:36] tlrobinson_ has joined the channel [04:36] kriszyp: thank you, didn't realize it would be that easy [04:37] isaacs: kriszyp: you could also man up and use git :P [04:37] isaacs: but that's less easy, i gues [04:38] kriszyp: yes, I could do that I suppose [04:42] kriszyp: arg, it looks like relative references in module requires are broken in master :( [04:42] isaacs: kriszyp: you have a failing test? [04:42] kriszyp: I can make one... [04:43] isaacs: ACTION is like a junkie for failing tests [04:43] eviltwin has joined the channel [04:44] mattly has joined the channel [04:46] jamiew has joined the channel [04:53] mikeal has joined the channel [04:58] kriszyp: isaacs: http://gist.github.com/271002 [04:58] kriszyp: and I am pretty sure that was working 0.1.23 [04:58] isaacs: looks familiar [04:59] isaacs: it works if you don't do it with the require.paths bit [04:59] isaacs: but that should be doing a path.normalize() anyhow [05:00] kriszyp: yeah [05:00] kriszyp: just normal relative references work fine [05:00] isaacs: can you verify that it works if you change require.paths.push("../folder2"); to require.paths.push(require("path").normalize(path.dirname(__filename), "../folder2"))); [05:00] kriszyp: it is definitely in combination with the relative paths in the require.paths that breaks it [05:00] isaacs: kewl [05:00] kriszyp: i'll try [05:01] kriszyp: am I supposed to be doing it that way? [05:01] isaacs: feel like adding that test to test/mjsunit and sending an email to the list with the patch including the test you added? [05:01] isaacs: nono, it's supposed to work [05:01] isaacs: i just wanted to make sure the bug is "relative paths in require.paths" and not "require.paths is broken" [05:02] isaacs: (don't feel obligated to add the test to node, it's just a nice reminder to us all that something is broken if "make test" fails) [05:03] kriszyp: not sure I follow, what is the path variable supposed to be? [05:04] isaacs: oh, nvm... [05:04] isaacs: the path var is the require("path") [05:04] isaacs: yeah, i'm reproducing it now, i'll add the test and just give you the credit for finding it ;) [05:04] kriszyp: thanks for the help [05:05] isaacs: np [05:07] isaacs: hm... actually... since require.paths is just an array, i'm not sure how that would ever work. [05:08] isaacs: should it work like the $PATH variable maybe? like, if you push "." onto it, then you can always load any local module as just require("foo")? [05:08] isaacs: kriszyp: ^ [05:08] kriszyp: I thought so [05:09] kriszyp: but I don't know [05:09] kriszyp: I don't need the to push "." for my stuff [05:09] kriszyp: but lots of "../../somefolder" [05:09] isaacs: but you should be able to just do require("../../blah") [05:10] isaacs: what i'm saying is, should it resolve it when you push()? [05:10] isaacs: or should it resolve it relative to whichever module is calling require() at that time? [05:10] kriszyp: I think it should resolve it when you push [05:10] isaacs: kk [05:10] isaacs: either way, your test case should work, just just concerned about the implications. [05:11] kriszyp: I am using lots of narwhal modules, so I can't change them all to do require("../../blah"); [05:11] isaacs: any idea how other commonjs impls do it? [05:11] kriszyp: kriskowal: are you around? [05:11] isaacs: that is, i though that the commonjs spec was that require.paths should be an array of strings, not a magic resolving thing. [05:12] kriszyp: but the strings are paths, right? [05:12] isaacs: right [05:13] isaacs: but... let's say I do require.paths.push(".") [05:13] keeto has joined the channel [05:13] isaacs: now, require("path") and require("./path") are the same, no? [05:14] isaacs: like, it's not as if i'm tracking WHICH module pushed "." onto the paths, and then resolving it against what "." meant at that moment. [05:14] kriszyp: no, that would depend on where you are requiring from [05:14] isaacs: i'm just always looking for "./blah" whenever you do require("blah") [05:14] kriszyp: of course [05:15] isaacs: ok, then that's pretty easy, but it also seems scary to put relative urls in the require paths. [05:15] isaacs: just like it's scary to put "." in your $PATH, because now you can easily hijack "cd" by being in the same dir as a a "cd" executable [05:15] kriszyp: so the paths define the root folders that you are going look in for modules [05:16] kriszyp: but if you are inside a commonjs virtual nested folder (like inside require("folder/bar") when you do a require("./blah"), it is going to look in the "folder" directory inside each of the paths from require [05:17] hassox: lads.. how much influence does common.js have over node? [05:17] kriszyp: so if you require.paths.push(".") it would look in ./folder/ for blah [05:17] kriszyp: no one has to follow commonjs at all if doesn't want to [05:18] hassox: not quite what I mean [05:18] hassox: is node.js aiming for complience? [05:18] kriszyp: does that make sense isaacs? [05:18] hassox: and or what would cause ppl not to want complience? [05:18] isaacs: hold on... so, in folder/foo/bar, I do this: require.paths.push("."); then, way later, in folder2/asdf/quux I do require("bar"), and it looks for "./bar", which is folder2/asdf/bar [05:19] isaacs: oui? [05:19] kriszyp: require.paths.push(".") is not sensitive to what module you are in [05:19] isaacs: right. [05:19] isaacs: it's sensitive to where you are when you call require() [05:19] kriszyp: right [05:19] isaacs: wonderfl [05:20] isaacs: then we don't have to split the atom to fix this ;) [05:20] kriszyp: ACTION wonders if that was sarcastic [05:20] isaacs: no, i seriously would have had to split atoms otherwise. :P [05:21] isaacs: all joking aside, it's important to get that right, because it's that kind of subtle weird thing that leads to big confusing errors down the road [05:21] kriszyp: yeah [05:21] isaacs: i'm convinced that you've found a bug, but just want to make sure we're fixing it in the right way [05:21] kriszyp: right [05:22] kriszyp: sorry, I am having a hard time explaining [05:22] isaacs: no problem, it's a tad convoluted [05:23] isaacs: i see the issue [05:23] kriszyp: and I know that node also differs from narwhal (or has) in that when you do require("./blah") it only seems to look in the directory from which the current module is loaded instead of the paths. Which is fine with me, I don't mind the difference, but it was a little unnerving at first [05:23] isaacs: yeah, look at around line 797-809 of node.js [05:24] isaacs: http://github.com/isaacs/node/blob/542fbaf043652464c9d3781efbf2c7117069417d/src/node.js#L798-809 [05:25] isaacs: it checks whether or not it's relative, and if not, it checks all the require.paths, but it's always strictly either-or [05:25] isaacs: that's not ideal. [05:25] kriszyp: oh, didn't realize the code as right there, I guess I could have hacked on that myself... [05:29] isaacs: so it seems like the algo should be: if it's not "." or "/", then look through require.paths, resolving any relatives against the current dir. if it does start with ".", then check the current dir directly first [05:29] kriszyp: and then go to the require.paths? [05:29] isaacs: yeah. [05:29] isaacs: should be very light change. [05:30] kriszyp: cool [05:32] kriszyp: well, I need to get to bed, thanks [05:44] mikeal has joined the channel [05:55] eviltwin has joined the channel [05:59] bentomas has joined the channel [06:05] tlrobinson_ has joined the channel [06:10] alex-desktop has joined the channel [06:17] jamiew_ has joined the channel [06:32] isaacs: compilation makes me crazy [06:36] hassox: no beer and no tv makes hassox something something [06:40] hassox has left the channel [06:54] bentomas has left the channel [06:55] rictic has joined the channel [07:04] okito has joined the channel [07:24] scudco has joined the channel [07:37] mjard: go crazy? [07:37] mjard: DON'T MIND IF I DO [07:39] mjard: if I use process.mixin, that's going to be a global change, available across modules? [07:43] mikeal has joined the channel [07:46] CIA-78: node: 03Vanilla Hsu 07master * rd22952b 10/ (src/node.cc wscript): getmem for freebsd - http://bit.ly/7GgObZ [07:57] elliottcable: Argh [07:57] elliottcable: everybody I know is asleep [07:57] elliottcable: mikeal: to what? [07:57] mikeal: huh? [07:57] elliottcable: somebody! anybody! enjoy my success with me! [07:57] elliottcable: ACTION breaks out the inebrieating beverages [07:58] elliottcable: http://cld.ly/d1zjs [07:58] elliottcable: mikeal: what is going to be a global change… to what… across what modules? [07:58] mikeal: how long ago did I say that? [07:58] elliottcable: mikeal: ~20 minutes [07:58] elliottcable: aw crap mikeal [07:58] elliottcable: sorry… [07:59] elliottcable: that was for mjard >,< [07:59] mikeal: hehe :) [07:59] elliottcable: ACTION is high on success, and didn’t notice the nick [07:59] elliottcable: mikeal: well, whatever. Tell me how cool I am for having made this work! d-: http://cld.ly/d1zjs [07:59] ryah: elliottcable: real link? [07:59] mikeal: that the new repl? [07:59] elliottcable: ryah: eh? [07:59] elliottcable: mikeal: not yet [08:00] ryah: ACTION can't resolve cld.ly [08:00] elliottcable: mikeal: I just finished the prototype for a ‘styled String’ that knows how to print itself to the terminal, intelligently [08:00] elliottcable: ryah: DNS hasn’t propogated; the cld.ly guys tell me they changed it today [08:00] elliottcable: ryah: works for me, on Google Public DNS [08:00] elliottcable: mikeal: but the pretty printer (Percival) comes tomorrow, and the repl is already set up for me to drop that in place [08:01] mikeal: nice [08:02] elliottcable: mikeal: did cld.ly resolve for you? [08:02] mikeal: yeah [08:02] mikeal: i also use the google dns [08:03] mikeal: it's actually faster than Comcast's [08:03] elliottcable: mikeal: ’ere’s the source that printed that string: http://cld.ly/56zju [08:04] ryah: god. news.ycombinator.com doesn't end it's http lines with \r\n only with \n [08:05] ryah: s/it's/its/ [08:05] elliottcable: o_o [08:05] elliottcable: isn’t that fairly evil? [08:06] ryah: curl -I http://news.ycombinator.com 2> /dev/null | hd [08:06] elliottcable: ryah: that’s xxd on OS X, fyi [08:06] elliottcable: ryah: what platform are you on, anyway? [08:07] elliottcable: ugh I keep typing `.exit` into zsh d-: [08:07] ryah: linux [08:07] elliottcable: ryah: what flavour? [08:07] ryah: ubuntu [08:12] mikeal: i'm still seeing this stdio read issue with the latest master [08:13] mikeal: the read event just isn't firing [08:14] mikeal: sorry, the "data" event [08:15] ryah: mikeal: did you post about this before? sometimes message slip by me [08:15] mikeal: yeah, i posted a test to the list [08:15] mikeal: lemme find it [08:15] mikeal: http://groups.google.com/group/nodejs/browse_thread/thread/10fda8eaf7276642/febbe555dd7fa95e?lnk=gst&q=test+stdio#febbe555dd7fa95e [08:16] mikeal: it's entirely possible that the bug is only on Mac [08:18] ryah: hm [08:18] michaelk^ has joined the channel [08:18] mikeal: i'm compiling on my debian box now [08:18] ryah: yeah it works for me on linux [08:18] mikeal: damn, it might be a kqueue bug on Mac [08:18] mikeal: i keep reading about how everything except sockets are too buggy to use on Mac [08:19] ryah: works on mac too [08:19] ryah: or at least my mac [08:20] mikeal: node test.js [08:20] mikeal: works? [08:20] ryah: outputs "success" anyway [08:20] ryah: er wait a sec .. clearly i need to update the path [08:20] mikeal: that means it worked [08:22] ryah: yeah seems to work for me [08:22] mikeal: print the data it reads from output [08:22] mikeal: just to make sure it's not an exception [08:26] mikeal: on debian the event fires but it's null [08:27] ryah: okay i get the error now [08:30] ryah: mikeal: want to clean this up into a test/mjsunit test? [08:30] ryah: :) [08:31] mikeal: sure [08:32] ryah: mikeal: http://gist.github.com/271086 [08:32] ryah: might help [08:33] mikeal: it does [08:35] tlrobinson_ has joined the channel [08:35] johan-s has joined the channel [08:39] DamZ has joined the channel [08:45] mikeal: ryah: http://github.com/mikeal/node/commit/e91db132c0d0a7f91d0dd94319480417015fd098 [08:46] ryah: mikeal: can you put echo.js into fixtures [08:46] mikeal: sure [08:46] ryah: mikeal: make sure you run all the tests [08:47] ryah: i think the listdir test needs to be updated [08:47] ryah: also, i think data could be null sometimes [08:47] ryah: on eof [08:47] ryah: so doing data[0] will be an error when it works [08:47] mikeal: ahh [08:48] mikeal: i'll do a type check [08:48] ryah: if (data && data[0]) [08:48] botanicus_ has joined the channel [08:48] ryah: mikeal: assert.ok(result) [08:49] ryah: mikeal: don't need sys.puts() just do "puts" [08:49] ryah: oh you do.. don't require('sys' [08:49] ryah: otherwise looks good [08:50] mikeal: ok [08:50] ryah: remember ../common in echo.js when it's in fixtures [08:50] ryah: thanks [08:50] ryah: i'll try to do something with that tomorrow [08:51] mikeal: yeah, just fixed [08:51] mikeal: all pushed to my master [08:52] felixge has joined the channel [08:53] ryah: mikeal: squash it and email me the patch please ry@tinyclouds.org [08:53] mikeal: hehe [08:53] mikeal: ok [08:53] felixge: ryah: morning :) [08:53] ryah: felixge: hey [08:53] mikeal: what format do you want the patch in? [08:53] mikeal: just attach the two files? [08:53] ryah: mikeal: git format-patch format [08:54] felixge: ryah: posix.readSync() seems to only return the # bytes read, not the chunk itself [08:54] ryah: felixge: hmm not an array? [08:55] ryah: felixge: ah, true [08:55] felixge: ryah: when changing promises to emit a single value I uses: {chunk: chunk, bytesRead: bytesRead} [08:55] felixge: * used [08:58] ryah: http://s3.amazonaws.com/four.livejournal/20100107/read_sync.diff [08:58] ryah: you might have to hack that a bit [08:59] felixge: ryah: perf., np [08:59] ryah: oops [08:59] felixge: ryah: I think I'll have a demo for you to look at it a bit, how much caffeine do you have in your blod? :) [08:59] felixge: * blood [09:00] ryah: i'm drained - going to bed in a second :) [09:01] ryah: felixge: http://s3.amazonaws.com/four.livejournal/20100107/read_sync2.diff [09:03] felixge: ryah: ok, fair enough. You should have a require() sync patch or some stupid question in your inbox by the time you wake up ;) [09:04] ryah: cool ) [09:06] teemow has joined the channel [09:18] elliottcable: g’nite ryah [09:19] DamZ has joined the channel [09:21] mikeal has joined the channel [09:34] caolanm has joined the channel [09:40] caolanm: is there a nice way of catching errors that occur in an asynchronous function? ...say I have a function that calls posix.cat and then does something in the callback, what would be the nicest way of having that function 'catch' errors that occur in the callback? [09:41] caolanm: best option I can think of is wrapping the callback in try/catch and calling errback with the error... is that right? [09:50] mayerbacher has joined the channel [09:52] felixge: caolanm: you have to do the wrapper inside the callback [09:53] felixge: caolanm: once the interpreter has moved outside a try..catch block, its not going to catch even if a callback takes you back in there [09:54] caolanm: felixge: yeah, it looks like my issue is similar to this one: http://groups.google.com/group/nodejs/browse_thread/thread/8c8d9a59a8280fd2 [09:54] felixge: caolanm: but we're trying to improve upon the promise API [09:54] caolanm: good, it feels a bit clunky doing things this way ;) [09:54] felixge: its just a big bikeshed discussion and takes a bit longer to sort out than it should [09:54] DamZ has joined the channel [09:55] caolanm: felixge: the problem is that a callback outsite my try/catch could actually crash my http server [09:56] felixge: caolanm: anyway, thanks for bringing this up again - it makes me more confident that this is a real problem we are solving here [09:56] felixge: caolanm: yeah, for now your best workaround is process.addListener('uncaughtException') [09:56] felixge: this allows you to swallow all uncaught exceptions [09:56] felixge: but it sucks because you'll be taken away from the context your exception occured in [09:56] felixge: so you may not be able to do something useful with it easily [09:57] caolanm: ok, so at least I can stop the server crashing, even if it means I can't return a sensible response... [09:57] elliottcable: ack [09:57] elliottcable: does sys.puts() not toString things before displaying them? /-: [09:57] caolanm: felixge: thanks btw :D [09:58] felixge: elliottcable: no, but it should - patch time ;) [09:58] elliottcable: hrm [09:58] elliottcable: I was fairly sure it did [09:58] felixge: caolanm: np [09:58] elliottcable: hold on, I think I have a bug locally [09:59] felixge: well, it does: x + "\n" [09:59] felixge: so JS should have an incentive to use toString() [10:00] elliottcable: but yeah if it doesn’t, I’ll certainly fix that [10:00] elliottcable: ACTION hacks [10:07] felixge: ok, sync module loading is a bitch [10:07] felixge: not doing things async is really hard in node :D [10:13] inimino: that's a good sign :) [11:56] felixge has joined the channel [12:35] tisba has joined the channel [12:43] spoob has joined the channel [12:52] unom1 has joined the channel [12:57] jakswa has joined the channel [12:59] charlenopires has joined the channel [13:00] jakswa: hello world works! and it is glorious... [13:03] mediacoder: and that was JUST the hello world..imagine how awesome the rest is ;-) [13:10] caolanm: I sort of expected an error in a Promise callback would call the errback, so that errors could bubble up through a chain of promises... am I misunderstanding how promises should be used? [13:11] alex-desktop has joined the channel [13:19] pmuellr has joined the channel [13:32] hornbeck has joined the channel [13:33] jed has joined the channel [13:36] dnolen has joined the channel [14:10] cloudhead has joined the channel [14:12] inimino: caolanm: you mean exceptions? [14:12] inimino: caolanm: if you want exceptions to be caught, then you have to catch them [14:23] jakswa has joined the channel [14:24] jakswa: i wonder if anyone would mind me starting the wikipedia page for this... would be my first wikipedia page ever ^_^... but i feel it's necessary... [14:25] jakswa: nevermind, looks like it was attempted, and then deleted [14:27] inimino: probably not considered notable yet [14:31] bryanl has joined the channel [14:37] sveimac has joined the channel [14:37] davidsklar has joined the channel [14:41] kriszyp has joined the channel [14:45] robrighter has joined the channel [14:52] cloudhead has joined the channel [14:52] cloudhead_ has joined the channel [15:02] soveran has joined the channel [15:03] mahemoff has joined the channel [15:07] alexiskander has joined the channel [15:11] cloudhead has joined the channel [15:12] charlenopires has joined the channel [15:17] bentomas has joined the channel [15:27] mahemoff has joined the channel [15:36] ryah: reset [15:36] ryah: oops [15:41] eviltwin has joined the channel [15:43] DamZ has joined the channel [15:45] jan____: ACTION resets ryah [15:46] voodootikigod has joined the channel [15:47] mahemoff has joined the channel [15:47] qFox has joined the channel [15:57] spoob_ has joined the channel [16:13] binary42 has joined the channel [16:13] kriszyp: it seems that after some amount of file operations that the system gets in a state where posix.stat quits working; when i call posix.stat the callback is never called (nor the errback). Does anybody have any idea why? [16:18] kriszyp: and if I call process.loop() from the repl, all the queued up stat callbacks proceed [16:24] mahemoff has joined the channel [16:32] dnolen has joined the channel [16:33] spoob_: pumping the runloop is something I have to do in Cappuccino as well [16:33] bry has joined the channel [16:34] inimino: kriszyp: recent issue? [16:35] inimino: kriszyp: maybe something to do with all the libeio patches flying about at the moment? [16:35] martint-t has joined the channel [16:35] kriszyp: I don't know if it is recent [16:35] kriszyp: I had gotten this far in my project before [16:37] jspiros_ has joined the channel [16:37] kriszyp: spoob_: how do you know when to restart the event loop? [16:37] kriszyp: that doesn't seem like a good thing... [16:37] spoob_: when I do an update to the contents of a window, and it doesn't show. It's a similar but not related pronlem [16:38] spoob_: the event loop hasn't died, it just needs to be triggered when there's underlying changes to the contents that were not done by mouse movements [16:53] kriszyp: so it seems to be pretty easy to reproduce [16:54] kriszyp: http://gist.github.com/271364 [16:55] nsm has joined the channel [16:57] kriszyp: or alternately, this one only does 3 at a time concurrently, and for me is much more unpredicatable in how far it makes it: http://gist.github.com/271368 [16:59] nsm has joined the channel [16:59] inimino: kriszyp: this won't matter, but you're passing extra arguments to posix.stat() [16:59] kriszyp: oh, oops, yeah, I originally was doing posix.open [16:59] kriszyp: and just did s/open/stat [17:00] inimino: kriszyp: it only takes one [17:00] kriszyp: right [17:00] kriszyp: definitely still behaves the same with one parameter though [17:01] kriszyp: and with open (with the extra params) it fails to [17:02] inimino: hm, it got stuck at 15 here [17:02] kriszyp: try it again [17:02] kriszyp: it usually different for me each time [17:02] kriszyp: for the second test case anyway [17:02] kriszyp: the first one is more consistent [17:02] kriszyp: anyway, thanks for the verification [17:02] kriszyp: nice to know I am not crazy [17:04] inimino: 15 the first three times, then 26 [17:04] inimino: then 17 [17:04] kriszyp: yeah, feels like a race condition somewhere [17:04] spoob_: here a stat, there a stat, everywhere a stat stat, ol' mc.node had a crashy crashy :) [17:05] inimino: 26, 17, 16, 16 [17:05] inimino: yes, definitely [17:05] kriszyp: and note that if you only do one stat at a time, it works fine [17:05] RayMorgan has joined the channel [17:06] spoob_: the fact that you can do one pull and it kicks the others through suggests some kind of race condition where it has missed one notification [17:06] kriszyp: now if I could get all the users of my app server to coordinate so no HTTP requests overlapped, my app would probably run fine :) [17:06] spoob_: poll not pull [17:07] inimino: almost certainly a libeio-related issue, ryan was saying something very similar to this [17:07] inimino: make sure he sees this [17:07] kriszyp: yeah, sending it to the ml... [17:07] spoob_: ryan; php is better than node! OK, he'll see this :) [17:08] kriszyp: heh [17:10] hober has joined the channel [17:12] felixge has joined the channel [17:13] felixge: 2+ [17:15] isaacs has joined the channel [17:17] jed__ has joined the channel [17:17] jed__: just out of curiosity, is there a scenario in which sendBody is called more than once with different encodings (the second argument of sendBody, which defaults to "ascii")? [17:18] erichocean: yes [17:19] inimino: jed__: sure, if a template system were to concatenate files with different encodings, for example [17:22] jed__: inimino: i'm having trouble picturing how that'd work. what are the legal values for the encoding parameter? ascii, utf8, and binary? [17:22] inimino: jed__: yes [17:23] jed__: inimino: so if this template system concatted different encodings into a string, it'd still need to somehow mark where each one changes, right? [17:23] inimino: jed__: I'm thinking of a streaming system, where some parts may come from disk, some from strings in the source code, etc [17:23] inimino: jed__: and it is sending them out from different parts of the code in different places [17:23] inimino: jed__: I should have said "sending separately" not concatenating [17:24] inimino: (concatenated only from the perspective of the final outgoing result streaming out to the client) [17:24] jed__: so it'd be used for either adding binary to multi-part responses, or mixing ascii and utf8, right? [17:25] inimino: so something calls sendBody(current_date, "ascii") and then something else calls sendBody(file_from_disk, "binary") [17:26] inimino: well it could be anything interleaved with anything, if you have something like SSI or PHP-style spaghetti [17:26] jed__: in that case, what does node do to delineate those parts? [17:26] jed__: it just writes them straight in? [17:27] inimino: node doesn't do anything, the encoding parameter is just an argument to node to tell it what that data actually is [17:27] inimino: if it's "binary" the low byte of each UTF-16 code unit in the JavaScript string gets sent out over the wire [17:27] inimino: if it's "ascii" it's the low 7 bits [17:28] inimino: if it's UTF-8, then the UTF-16 data is encoded as UTF-8 before being sent out [17:28] jed__: inimino: and it'd up to the client to figure out where the encoding changes within a response? [17:29] inimino: it doesn't change within the response [17:29] inimino: once those things are sent out they are just bits on the wire [17:29] inimino: they could be in any encoding, they could be binary data, or they could be ... anything [17:29] jed__: right, what i meant is that it's up to the client to interpret it. [17:29] isaacs: i was kinda wondering that myself. i presume it's just bytes over the wire, but if i want to send "schlüter" to the client, what encoding should i put in the content-type? [17:30] inimino: the handling of the encoding argument is entirely internal to node and how it interprets the string argument passed along with it [17:30] inimino: right [17:30] inimino: put UTF-8 in the content-type [17:30] jed__: for charset in the content-type header, you mean? [17:31] isaacs: so, i've been calling sendBody(data, "binary") and a utf-8 encoding header [17:31] isaacs: jed__: yeah [17:31] isaacs: that seems to work fine [17:31] inimino: and then use "utf-8" in the encoding argument to node, if you have that in a JavaScript string (which will be UTF-16) [17:31] inimino: yes [17:31] isaacs: inimino: so, if i send it as "binary", then it's actually sending utf-16 bytes, right? [17:32] jed__: inimino: okay, thanks for clearing that up. [17:32] isaacs: (ie, bytes which represent characters when decoded according to utf16, for the technical hair-splitters in the audience) [17:32] inimino: if you have data that you got from reading a binary file on disk, and that file is in UTF-8, then you should use "binary" and you'll be sending UTF-8 [17:32] isaacs: right [17:32] isaacs: but what if it's a javascript string [17:33] isaacs: sendBody("schlüter", "binary") // this is actually sending utf16, which just happens to be the same as utf8 in this case, yes? [17:33] inimino: isaacs: if you write sendBody("schlüter","binary") then that won't work well [17:33] jed__: it seems like there's an opportunity for a more intuitive API here: since there's a setBodyEncoding method on request, it'd be better to have the same method on response, and use that instead of a second argument. [17:34] isaacs: ryah_away mentioned that it would be nice to use utf8 everywhere, but binary is faster, and ascii is WAY faster. [17:34] inimino: jed__: yes, I was talking with Ryan about overhauling the API yesterday or so, we're all agreed that there's room for improvement [17:35] inimino: yes, performance reasons, currently [17:35] inimino: but I think with the stuff he's doing in net2.js and the new external byte arrays V8 seems to have, we can clean a lot of this up [17:35] isaacs: that's awesome [17:35] isaacs: just throwing everything into utf8 and forgetting about it solves SO many annoying problems. [17:35] jed__: very cool, looking forward to that. [17:35] isaacs: i'd be ok with utf16 as well, but it seems like it's more bloated and less frequently used. [17:36] inimino: in particular, I want explicit UTF-8 ↔ UTF-16 conversion routines [17:36] inimino: and then all the I/O functions can just deail with bytes, and maybe have fooAscii variants [17:36] isaacs: right. [17:36] isaacs: i actually rather like narwhal/commonjs's BinaryString stuff. [17:36] isaacs: makes it much more explicit. belongs in javascript proper, imo [17:37] inimino: we are more or less stuck with some level of UTF-16 since it's required for JavaScript strings [17:37] inimino: yeah, definitely [17:37] sveimac has joined the channel [17:38] isaacs: so, if v8 has new byte arrays that are winful, maybe BinaryString could be implemented on that? [17:38] inimino: and it should get added soon, I know it's a major priority for at least one browser vendor for the next round of ES-spec work [17:38] inimino: and WebSockets also badly need it [17:38] isaacs: yeah, definitely [17:38] inimino: yeah, I think they definitely could be [17:38] isaacs: ACTION cheers [17:39] mahemoff has joined the channel [17:39] jed__: inimino: one more thing. the docs currently say "If chunk is a string"... are there any other valid types? [17:39] jed__: (for sendBody) [17:39] jed__: or is this just forward-loooking for an eventual BinaryString? [17:40] inimino: jed__: that's a holdover from when it used to allow arrays of integers as a "raw" type [17:40] inimino: it's backward-looking and should be fixed in the docs :) [17:40] eddanger has joined the channel [17:40] jed__: inimino: ha, okay. the docs are pretty far out of sync with master, lemme know if i can help when it's time to update them, okay? [17:41] isaacs: jed__: if you see a discrepancy, i think no one will mind if you update them now and send a patch. [17:41] isaacs: in my experience, doc updates are *always* welcome and very seldom provided. it's not as sexy as hacking on code, so it's less enticing. [17:41] inimino: yeah, docs patches are always welcome [17:41] isaacs: same with tests [17:41] isaacs: actually, imo, a failing test is MORE valuable than working code. [17:42] jed__: so the docs are intended to be in sync with master, and not the latest tag or anything? [17:42] inimino: exactly [17:42] isaacs: jed__: that's why they're in git :) the docs on a tag should be in sync with that tag. [17:43] jed__: and the docs that show up on http://nodejs.org/ are kept in sync with master? [17:43] inimino: the docs on the site are generated from the current stable release, but yeah, what isaacs said [17:43] isaacs: jed__: i believe they're kept pulled from the same tag as the release version, in that case. [17:43] jed__: ah, okay. [17:43] isaacs: so that they match the code in the download link right there. [17:43] jed__: in which case they're up to date. [17:44] jed__: (if the latest tag is the "stable version", that is) [17:44] isaacs: but if you're building from git, i find it helpful to refer to the docs in git, too, just to be aware of those kinds of issues. [17:44] isaacs: if you install "asciidoc" then you can even look at them in pretty html :) [17:44] jed__: ah, i'll do that now. [17:44] jed__: thanks for the tip. [17:44] isaacs: np [17:45] isaacs: i'm having some issues with the asciidoc textmate bundle, though [17:45] isaacs: whenever it reads a file as asciidoc, it seems to have a tendency to peg my cpu and freeze up [17:49] jed__: so the workflow would be, fork node, install node from your fork, update the docs there, push, and then send someone a github pull request or something? [17:49] jed__: i'd like to avoid having two separate node repos if i can. [17:50] jed__: (although i could be doing this all wrong, i'm not really a git-wizard.) [17:50] isaacs: jed__: just do it on a branch [17:50] jed__: isaacs: i see. and once i push to that branch, how do i notify? [17:51] isaacs: jed__: (assuming your "master" branch matches ry/master) git checkout -b doc-fix-for-blah; (... fix docs, test, commit ...); (push to github); (email nodejs "hey, guyz, i fixed the docs for blah! get it at on my doc-fix-for-blah branch) [17:52] isaacs: jed__: email the list. there's lots of patch/bugfix emails there if you want to match the style that seems to be evolving, but i don't think there's any particular standard. [17:52] jed__: isaacs: github pull requests are okay? that's probably easier/less noisy. [17:53] inimino: jed__: you can also just use git format-patch and send it to ry@tinyclouds.org (or the list) as an attachment [17:53] inimino: (if that's easier) [17:53] isaacs: jed__: not sure. in my experience, whenever i get a pull request, i just end up doing it manually anyhow, and i think having it on the list is good because you can get more eyeballs on it. [17:53] isaacs: especially doc bugs, since that might hilight a problematic api or something [17:56] jed__: cool. this meta-info would probably be helpful on the github wiki too. [17:56] scudco has joined the channel [17:56] isaacs: jed__: if you ever learn something that you think would've been useful for you to have known already, feel free to write it down for future people. [17:56] isaacs: everyone else probably didn't record it because they're lazy, not because it's a secret. [18:01] isaacs: ACTION is often part of the lazy "they" above, not meant as snark, just statement of fact [18:01] jed__: isaacs: don't worry, i lazily evaluated that statement. [18:02] isaacs: hehe [18:02] jed__: aight, time for lunch. thanks for your help isaacs and inimino. [18:02] isaacs: have fun! [18:10] stephenlb has joined the channel [18:22] DamZ has joined the channel [18:25] ericflo has joined the channel [18:26] jamiew has joined the channel [18:30] jcrosby has joined the channel [18:30] okito has joined the channel [18:44] tav has joined the channel [18:44] tav: ACTION waves [18:44] tav: hey guys, just started playing with node — looking nice! [18:44] felixge: isaacs: how is your setExports() bikeshed coming along? [18:45] felixge: I'd kind of like it and I'm sure everybody is weak from the discussions so we could just ask ryan to accept a patch :) [18:45] isaacs: felixge: ugh [18:45] isaacs: exhausting [18:45] felixge: isaacs: any real issues with it other than nay sayers? [18:45] isaacs: nope [18:45] isaacs: not that i can tell. [18:46] felixge: well lets fuck it then :) [18:46] isaacs: there are a few problems that we already have, which it doesn't solve. [18:46] Booster has joined the channel [18:46] isaacs: (where "we" == "every commonjs platform") [18:46] isaacs: i do need to rebase my implementation on top of ry/master [18:46] felixge: yeah ... [18:46] felixge: well I think I have 1 big issue with CommonJS [18:47] bentomas: speaking of modules, is there anyway to pass variables to a module? I'd like to be able to include a module that does stuff but pass a configuration object to it. [18:47] felixge: initially I was really excited about the ability to write code that would run on the server and client [18:47] felixge: and having it compatible [18:47] felixge: but actually ... I have not done that so far, so its not really something I need [18:47] isaacs: felixge: there still are cases like that. [18:47] felixge: right [18:47] felixge: but ... I don't care [18:47] isaacs: hehe [18:47] felixge: I'll not go there [18:47] felixge: All those client site engines suck [18:48] felixge: if I need to share code with my server, it will be most likely validation rules [18:48] isaacs: the main recurring issues, i've found, are form validation and business logic [18:48] felixge: and partial snippets here and there [18:48] felixge: but not a whole app [18:48] isaacs: but those are really annoying, and likely to change frequently [18:48] bentomas: the only way I can think of doing it is having the module return a function that takes the configuration object, and then that function basically returns what it wants to export. but that is really ugly [18:48] felixge: so I feel standards that are made with the browser in mind are actually making life much more difficult [18:48] isaacs: bentomas: sorry, didn't mean to ignore you.. [18:48] isaacs: yeah, you could also have the config object hang on the exports object directly, and always read it from that [18:49] felixge: bengl: you can't pass a variable directly to the module constructor [18:49] isaacs: then i could do var foo = require("foo"); foo.someConfig = 8; foo.doSomething() [18:49] felixge: * bentomas [18:49] felixge: yeah, that works [18:49] isaacs: where exports.doSomething = function () { require("sys").puts(exports.someConfig) } [18:50] isaacs: i actually do that with the querystring module. i couldn't think of a case where you'd want to override the default decode() and encode() functions, but i figured i'd expose it just in case you did, since it didn't really hurt anything to do so. [18:50] bentomas: isaacs: so that would work? [18:50] isaacs: bentomas: yeah [18:50] bentomas: isaacs: perfect! [18:50] isaacs: because the exports object is the SAME object that gets returned from require() [18:51] isaacs: (yet another case where the other module-proxying ideas in the setExports thread will fail, but setExports will succeed) [18:51] bentomas: and re:module.setExports() I agree. I don't think the browser arguments are irrelevant, because as they stand browsers still can't import a module currently. [18:52] isaacs: right [18:52] bentomas: and I am doing var Obj = require('obj').Obj; ALL OVER THE PLACE [18:52] isaacs: the caja guys like to think about the browser, because that's the target they're heading for. i get it. [18:52] isaacs: but i don't see why it should prevent this, which is essentially a very very convenient bit of sugar, and actually safer than just replacing module.exports. [18:53] bentomas: (sorry that was supposed to be "relevant" not "irrelevant") [18:53] bentomas: yeah [18:53] isaacs: (replacing module.exports works now, btw. you can do that. and my implementation adds a setter around module.exports, so it'll have the same protection as calling module.setExports) [18:55] felixge: isaacs: you may want to hold on with rebasing - I just send ryan a patch this morning that make require() sync [18:55] isaacs: bentomas: so, if you want to avoid your var Foo = require("foo").Foo [18:55] isaacs: felixge: yeah, i don't want to touch anything module/require-related until your hot/sync/async stuff is a bit more settled. [18:55] bentomas: felixge: that didn't take you very long! [18:55] felixge: bentomas: its pretty simple really [18:56] felixge: bentomas: I didn't make it very pretty yet [18:56] mikeal has joined the channel [18:56] felixge: just tried to keep redunancy low [18:56] felixge: I'd like to refactor the whole module loader a bit [18:56] felixge: depends on what ryan thinks [18:56] Sembiance has left the channel [18:56] felixge: http://github.com/felixge/node/commits/require-sync [18:57] isaacs: felixge: my change revolves almost entirely around constructing the "module" object which is exposed to the module. [18:57] isaacs: i suspect that once your stuff is done, it'll be pretty easy to get in there and make the setExports change. [18:57] felixge: isaacs: yeah, my patch just touches lots of code [18:57] felixge: and git does not understand diffs [18:57] isaacs: hehe [18:57] felixge: it just pretends to do so somewhat [18:57] felixge: ;) [18:58] isaacs: does a better job than cvs or svn, so i like it [18:58] isaacs: i'm told that perforce does like everything and can write your whole program for you [18:58] felixge: huh [18:58] isaacs: but you have to do a lot of p4 a x p --f s923 commands and shit [18:58] felixge: I wondered how this property vcs shit sells [18:58] charlenopires has joined the channel [18:58] felixge: I guess understanding the text it versions would be awesome [18:58] felixge: I digg [18:59] isaacs: felixge: i'm mostly told that by people who are either selling p4, or have never used anything else, or have used p4 for 5 years and git for 5 minutes, etc. [19:00] felixge: isaacs: yeah ... my understanding of lots of technologies is based on a similar information situation - kinda sad, makes it easy for me to dismiss stuff that might be cool ;) [19:00] felixge: like postgres :D [19:02] isaacs: people do love postgres [19:03] felixge: I do lots of CakePHP, almost nobody is using postgres with it [19:03] isaacs: for me, i've never been 100% sold on sql. i like the DSL, and thinking of data in terms of sets is neat, but the actual implementations are almost universally slow and have wonky issues. [19:03] felixge: and I've heard a few people tell me horror stories about postgres [19:03] rictic has joined the channel [19:03] isaacs: they're magic enough to be scary, but not magic enough to be wonderful. [19:04] felixge: yeah [19:04] felixge: well dirty will fix that [19:04] felixge: hoping to release dirty 0.1 tonight :) [19:04] isaacs: ahh, the "___ will fix that" meme... [19:04] felixge: A key value store so dirty people will fall for it :) [19:04] charlenopires_ has joined the channel [19:04] felixge: haha [19:05] isaacs: i want this to be crazy fast: select user.id, user.name from (select * from people where people.id in (select id from recent_users)) as user [19:05] dnolen has joined the channel [19:06] isaacs: you can DO that in most sqls, but the result is like a jillion stats and lookups and crazy lag and whatnot. [19:06] isaacs: once you optimize it for production use, it's more like "select data from table where key = 'blah'" [19:07] isaacs: and at that point, why not just use a doc store? [19:07] isaacs: good for offline/slow analytics, though [19:07] felixge: I think the part I like the most about sql is the schema constraints actually [19:07] felixge: its like using a framework [19:07] isaacs: right [19:07] felixge: people have to agree on conventions for the data format [19:08] isaacs: that reduces iterability, though [19:08] felixge: where with nosql, collaboration on stored data becomes not that obvious [19:08] felixge: well, no [19:08] felixge: it becomes a problem with production systems [19:08] felixge: but during development, renaming columns is very easy [19:08] isaacs: which is good, for some things, but all to often, people just use whatever tool they know, regardless of whether or not it's actually good. [19:08] felixge: true [19:09] isaacs: on a team of developers, "there's a schema update in this commit" is interpreted roughly as "sorry, guys, i might have accidentally pooped in the coffeepot, you might wanna go to starbucks today instead..." [19:10] isaacs: there are ways to mitigate that, of course. [19:10] felixge: heh [19:10] isaacs: you could put the bathroom and the kitchen far from one another so people are less likely to get confused. have poop-sniffing dogs by the coffee maker. maybe have the door lock when someone who has to take a dump is walking by. etc. [19:11] felixge: well not having a schema isn't going to make the developers old test documents he has locally work with the new code [19:11] felixge: maybe I'm not getting your point [19:11] felixge: but I think sql is at a slight advantage over key-value stores here [19:11] isaacs: but it's a bit more straightforward to put the docs in a vcs, or have changes tracked in some other way, or just accept that it's wonky and code to be robust, rather than relying on a schemal [19:12] felixge: as it makes it provides some guidance in migrating an existing data set [19:12] inimino: there are lots of amazing things about SQL [19:12] inimino: but there are tons of people doing things with SQL that have no business being done with SQL [19:12] felixge: true [19:12] isaacs: yeah, absolutely [19:12] isaacs: it's like a hammer vs a shotgun. using one for the other is Very Bad. [19:13] alex-desktop has joined the channel [19:13] inimino: it also seems to be something people feel they should "pick up" after PHP or whatever... [19:13] inimino: and then you get, well, pretty much every PHP-based software project you can name... [19:15] jed_ has joined the channel [19:15] inimino: the enthusiasm for nosql catching on with Web developers might be one of the best things to happen to both Web development and SQL [19:16] felixge: I have one huge problem with nosql [19:17] felixge: it solves a problem 95% of all web developers don't have [19:17] felixge: performance [19:17] orlandov: aye [19:17] felixge: that sounds weird, but most websites and apps are not in the top 10.000 [19:17] felixge: sure, everybody wants to be there [19:17] felixge: and nosql seems attractive as its "future proof" [19:18] felixge: but boy can you screw yourself missing out on development speed you get with sql [19:18] inimino: true, but the cost to the ones that are of rearchitecting everything is significant [19:18] felixge: and all those sql based companies figured out how to work around their growing pains eventually [19:18] felixge: and that's where nosql becomes interesting [19:18] orlandov: i feel like SQL gets a bad rap... people throw shit into a database without really thinking about what they're doing, and then they're surprised when it's slow [19:18] felixge: yeah, but I'd rather burn $10 mio. VC money rearchitecting than slaving over a scalable app that will never exceed 10 users [19:18] felixge: ;) [19:19] inimino: every few months somebody does a demo of some technology that would cut Wikipedia's hosting or server costs in half, but it's impractical for them to start over [19:19] inimino: well, if you're really saving time by using SQL then by all means do so :-) [19:19] felixge: yeah [19:20] felixge: I still love the idea of writing my DB logic in JS so (couchdb) [19:21] inimino: :) [19:21] felixge: so let me continue working on that [19:22] cloudhead: felixge: what's dirty? [19:22] felixge: cloudhead: a memory-based node.js key-value store with support for append-only disc persistence [19:23] felixge: it should be incredibly fast because you share memory with the key-value store directly, no communication overhead [19:23] inimino: sounds nice [19:23] cloudhead: felixge: how does it compare to redis then? [19:23] felixge: cloudhead: probably 10x faster [19:23] cloudhead: Oo [19:23] felixge: I can set 14 mio key/value pairs on a JS object with v8 on my laptop [19:23] felixge: that's the hard constraint [19:24] cloudhead: oh right, it doesnt go through stdin [19:24] cloudhead: doesnt memcachedb work like that? [19:24] felixge: if I don't fuck up on the disc persistence it shouldn't go lower than an order of magnitude on that [19:24] felixge: cloudhead: no, memcache is tcp [19:24] felixge: like reddis [19:24] cloudhead: kk [19:24] cloudhead: interesting [19:24] felixge: but that's overhead [19:24] felixge: also, you can actually run myStore.filter(function(){}); on the whole thing [19:24] felixge: and it will be reasonably fast for millions of documents [19:25] cloudhead: nice [19:25] cloudhead: C? [19:25] felixge: and you won't have much more than that due to the fact that it only lives in memory [19:25] inimino: that is a feature I really want :) [19:25] felixge: you'll have to shard by the time you run out of memory [19:25] cloudhead: except you can't really replicate it [19:25] inimino: ACTION uses something sounding very similar but very immature [19:25] cloudhead: like, it has to be local [19:25] felixge: cloudhead: no, its in JS completely [19:25] cloudhead: to your app [19:25] felixge: cloudhead: yes [19:25] felixge: but the file format will be dead-simple [19:25] felixge: newline terminated JSON [19:25] cloudhead: cool [19:26] inimino: you can split up your app, by URI subspaces, or whatever, and run node on different machines and load-balance them [19:26] felixge: yeah [19:26] felixge: well I'll make every additional feature a plugin [19:26] felixge: that overwrites the main modules functions [19:26] felixge: dirty - hell yeah ! [19:26] felixge: (that's the name of the key-value store) [19:26] cloudhead: interesting [19:26] felixge: should be fun, lets see if it works out [19:27] inimino: and use a SAN or something for shared persistence layer if necessary [19:27] inimino: cool :) [19:27] felixge: SAN? [19:28] inimino: some kind of network mounted filesystem [19:28] inimino: SAN = storage area network [19:28] cloudhead: felixge: so the data structure will be loaded into a js object essentially [19:29] felixge: inimino: that would only work if writes are atomic between different apps accessing it, but yes [19:29] felixge: cloudhead: yeah. Either that or an array [19:29] felixge: with an object for the index lookup [19:29] felixge: will depend on v8's profile [19:29] cloudhead: kk [19:29] felixge: binary42 said v8 is slow with for (var key in obj) for millions of records [19:29] cloudhead: arrays in js are a little slow [19:29] felixge: which would suck for views [19:29] cloudhead: yea [19:29] cloudhead: hmm [19:30] inimino: felixge: yes [19:30] binary42: Well, it wasn't me directly but @cramforce IIRC. [19:30] felixge: binary42: huh, thought it was you [19:30] binary42: Well, we discussed it but he discovered it. [19:30] felixge: ah ok [19:30] felixge: well, if its really like he reports, it's likely a v8 bug [19:31] felixge: which they would/should fix [19:31] felixge: and chunk the iterations like you said [19:31] binary42: It serializes keys before iteration it seems so only use it when key count is low. [19:31] inimino: felixge: did you write up your idea about streams yet? [19:31] felixge: inimino: no, its still more a gut feeling about what makes the shared ground, doesn't feel solid yet [19:32] binary42: btw: http://groups.google.com/group/nycjs/browse_thread/thread/e886873e84fa5a16 [19:32] binary42: For all of you near NYC. [19:32] inimino: k [19:33] felixge: binary42: it would be fun if stuff like coffee script could be hooked into the module loader [19:33] felixge: binary42: I'm 300-500 EUR near NYC, depends on the weekend :) [19:33] binary42: I don't think it'd be hard to make it work if it were ported to OMeta. [19:34] cloudhead: do you have to use C++ to interface with V8, or will C work? [19:34] binary42: felixge: Well, second Thurs every month. I'm also setting up a similar event in Brooklyn soon. [19:34] felixge: binary42: very cool! BerlinJS is going to meet on tuesday for the first time [19:34] felixge: and I'm doomed to talk about node.js, not fun after ryan's awesome talk :) [19:35] binary42: felixge: I'll totally try to make it to Berlin.js next time I'm around. [19:35] binary42: Might spend a month in Berlin later this year. [19:35] inimino: cloudhead: you would need a C++ layer for the V8 types but it could be pretty thin [19:35] felixge: cloudhead: v8 only has C++ interfaces, but ryan uses C libraries for node [19:35] cloudhead: ok good to know [19:36] felixge: binary42: I can provide a couch for a week or so if you're planning on couch surfing [19:36] jed_: binary42: thanks for that link, i think i'll be able to make it. [19:36] binary42: jed_: awesome. [19:36] binary42: felixge: Well, I'm going to try to sublet my place here so I can sublet in Berlin. [19:36] felixge: binary42: hah, your NYC place should by you mansion in berlin :) [19:36] binary42: Couch surfing sounds fun too but 1 mo. is a bit too long for that. [19:37] felixge: * buy [19:37] felixge: binary42: anyway, would be great to have you here! [19:37] binary42: felixge: Yeah. Though the € kills half that. [19:39] felixge: binary42: yeah, thank god for our half-ass chancellor who knows how to stay out of trouble ;0 [19:40] felixge: ACTION not a merkel fan [19:40] felixge: actually it kind of sucks cause we can't sell our fancy cars to reckless drivers no more [19:40] felixge: :( [19:41] binary42: hahah. I can't complain too much though. Last visit to Berlin ended with over 50% more cash left over than I expected. [19:42] felixge: hah :) [19:42] felixge: berlin is amazingly cheap compared to most other major cities in the world [19:43] binary42: That was even after purchasing a good portion of this: http://drp.ly/aNC8m [19:43] felixge: brianm: feigling or jägermeister? [19:44] binary42: feigling IIRC. [19:44] binary42: (That was what I was doing the night before my flight out at least) [19:44] felixge: hah [19:45] felixge: my new years saw lots of those as well [19:46] binary42: Hmm. Actually I think it was something else now that I look at my photo label. I think I misspelled it though. Looking it up [19:46] binary42: Kuemmerling. [19:47] felixge: ah [19:47] felixge: you're right [19:47] felixge: I think feigling doesn't even come in those bottles [19:47] binary42: Yeah. [19:49] bentomas has left the channel [20:01] brianm: felixge: ? [20:02] felixge: brianm: \n ? [20:02] brianm: ah, i think you meant binary42 [20:02] binary42: :-) [20:02] felixge: brianm: oh, sorry [20:02] brianm: no worries! [20:02] brianm: sorry to not read enough context to pick it up :-) [20:03] felixge: : ) [20:03] felixge: the js community is full of nice people [20:03] binary42: brianm: The real question is if you have an answer though. feigling or jägermeister? [20:03] felixge: hehe [20:06] brianm: old jubilation [20:08] isaacs: whoa, when did github add per-line comments!? [20:09] felixge: I present, dirty 0.0: http://github.com/felixge/node-dirty/blob/master/example/basics.js [20:09] felixge: isaacs: 10 internet years agfo? [20:09] isaacs: that's awesome! i thought it was just per-commit. never noticed the little comment bubbles in the margin before. [20:09] felixge: :) [20:10] felixge: yeah its nice [20:10] felixge: I wish I could also comment on whole files at a given revision [20:10] felixge: sometimes the patch context is too small [20:10] isaacs: yeah [20:10] isaacs: or if there were a way to expand the patch view. [20:10] isaacs: "show 5 more lines..." or something [20:10] felixge: that would actually be cool [20:10] felixge: but probably not easy [20:11] felixge: given how git stores commits [20:11] isaacs: well, yeah, but you can just take the patch and match it against the file's state at the parent commit [20:11] isaacs: it's slightly more work, but not rocket surgry [20:11] felixge: yeah [20:12] felixge: but github hasn't evolved much until they hired a designer [20:12] felixge: kind of cool : ) [20:12] isaacs: there's an internal code review tool at yahoo that does something somewhat similar to that, but for svn. [20:12] felixge: * until [20:12] felixge: I see [20:12] n8o has joined the channel [20:12] isaacs: actually, i think it's open source, and just modified slightly to work here. [20:14] DamZ has joined the channel [20:18] eviltwin has joined the channel [20:28] lattice has joined the channel [20:28] brianm: oh? [20:28] brianm: any idea the name? we used to use crucible but switched to git and crucible and git don't really get along that well [20:29] brianm: looking for anything that sucks less [20:29] brianm: oh wait, *for svn* [20:29] brianm: ignore me :-) [20:29] isaacs: hehe [20:29] isaacs: i think it's called "reviewboard" or something [20:29] brianm: oh [20:30] brosner has joined the channel [20:30] brianm: yeah, I kind of prefer needles under my nails [20:30] felixge: dirty benchmarks: http://github.com/felixge/node-dirty/commit/77b15cde8956ea5f86fa429153cb3c3032fc75a4 [20:30] felixge: 80k writes / sec in memory [20:30] felixge: 5k / sec on disk [20:30] isaacs: felixge: nice! [20:31] felixge: very likely have to "chunk" the writes [20:31] isaacs: yeah, ti's reviewboard: http://reviewboard.org/ [20:31] isaacs: brianm: ^ [20:31] isaacs: not sure if it's got git support, but if it does, it's worth checking out. [20:31] isaacs: very nice for async review-by-email/forum style stuff [20:32] isaacs: http://www.reviewboard.org/docs/manual/dev/faq/#does-review-board-support-git [20:32] felixge: http://groups.google.com/group/reviewboard/browse_thread/thread/fce7abb11375cac7 [20:32] brianm: yeah, have used it [20:33] felixge: yeah [20:33] brianm: needles under fingernails comment was in response to having used it [20:33] isaacs: yikes [20:33] isaacs: ACTION kind of hates the "let's all sit in a room with a projector and be unproductively talkative for 2 hours" style of reviews [20:33] isaacs: emailing patches is way better than that hell. [20:34] brianm: yeah [20:34] brianm: crucible (the atlassian thing) is actually really good [20:34] brianm: as long as you don't have too many repos -- it does some really inefficient tuff with lots of repos [20:35] brianm: we have lots of repos [20:35] brianm: I actually desperately want to do a nice async code reviewy thing which stores code review comments as, you know, comments [20:35] brianm: in the scm [20:35] brianm: as all too often some of the best commentary on code happens in the review [20:35] brianm: and is lost in the CR database [20:36] brianm: futzed with a markup for it some ( http://github.com/brianm/glitch ) but never got anywhere far [20:39] mattly has joined the channel [20:40] bentomas has joined the channel [20:54] inimino: brianm: that is an interesting idea [20:55] inimino: brianm: you could do all that in a collaborative text editor by just typing in comments in the code instead of chat [20:55] brianm: yep, you can interact directly through the repo, through tool designed to parse out the comments, or whatever [20:56] brianm: in my copious free time, never got very far :-( [20:56] inimino: heh [21:01] bentomas: does anyone know if it is possible to get the file location of a required module? [21:02] inimino: bentomas: other than by walking require.paths and looking for it? [21:02] bentomas: I could loop through the require.paths array to find the directory that contains the module and get it that way [21:02] bentomas: yeah other than that :) [21:02] inimino: then no :) [21:03] inimino: but you could look at what's there now and split that out into a "require.resolve(relative_path)" or something [21:03] bentomas: yeah, I'll think about it [21:04] inimino: probably would be a useful addition to the API [21:04] jed has joined the channel [21:08] lvmike has joined the channel [21:09] felixge: now we're talking (in terms of disk performance): http://github.com/felixge/node-dirty/commit/30118ec3a2008ffb02da8359485558e575a80844 [21:09] felixge: 99k writes / sec on disk is *pretty* awesome : ) [21:10] felixge: ACTION kind of suspects he is doing something wrong [21:10] felixge: ACTION but likely I just optimized my code to keep the little head on the disk from jumping [21:12] felixge: the speed limit for memory probably comes from JSON.stringify at this point [21:18] felixge: ok 60k writes / sec seems more realistic for bigger iterations [21:18] felixge: (to disc) [21:21] bentomas: felixge: boy what aren't you working on? promises, require.hot and now dirty! [21:21] felixge: bentomas: everything except client work and accounting for nov + dec [21:21] felixge: I'll be in trouble soon [21:21] felixge: ;) [21:22] felixge: actually getting some iphone development done for a client, but probably because that's new & cool :) [21:25] bentomas: nice! do you like iphone development? [21:26] felixge: bentomas: its love/hate [21:26] felixge: some of the best & most consistent APIs I've ever seen [21:26] felixge: but very verbose [21:26] bentomas: felixge: I think I'll stick to web development [21:26] felixge: writing: {foo: {bar: 'cool'}} in obj-c is 3 lines of code, no kidding [21:28] felixge: bentomas: give it a few more years and it will be all there is on mobile plattforms [21:28] felixge: even apple will move away from objective-c [21:28] bentomas: let's hope so! [21:28] felixge: but we'll need even better JS for that [21:28] felixge: :) [21:29] deanlandolt: felixge: better html and css for that ;) [21:29] felixge: deanlandolt: I don't love html/css, I actually like apples approach for laying out content in the apps [21:30] charlenopires has joined the channel [21:30] felixge: but web programming is huge, so people are working insanely hard to make it nicer [21:30] felixge: (take jQuery) [21:30] deanlandolt: fair enough...but to me js isn't really the bottleneck (though harmony does look nice)...it's browser capabilities...html5 is doing a pretty good job of fixing that though [21:31] felixge: deanlandolt: well, you'd be surprised how little resources you have available on the iPhone [21:31] felixge: I have to do lots of stuff in the background to make the experience smooth [21:31] felixge: pretty much all I/O [21:31] felixge: I wish there was node.js for the iPhone :) [21:32] felixge: deanlandolt: yeah, but rendering HTML/CSS would likely be a huge bottleneck [21:32] felixge: I agree [21:33] deanlandolt: i'm hoping folks start doing to canvas what jquery did to js...that should fix the bloat [21:33] deanlandolt: s/js/the dom/ [21:35] felixge: deanlandolt: have you seen http://raphaeljs.com/ ? [21:36] deanlandolt: felixge: yep...it's a hell of a start [21:36] deanlandolt: though it's more svg based i believe [21:36] binary42: :-) Yeah. It's not canvas. [21:36] deanlandolt: (i hear ms just joined the svg working group, so svg may /actually/ be usable soon) [21:36] binary42: http://glimr.rubyforge.org/cake/canvas.html#KeyboardTest [21:37] bentomas: I greatly prefer svg to canvas, but that's my personal preference. [21:37] bentomas: with svgweb by google you can use svg in IE pretty seamlessly these days [21:37] deanlandolt: i was disappointed when bespin dropped work on their thunderhead ui lib...hopefully someone comes around and works out all the accessibility kinks and puts together a kickass set of components [21:39] deanlandolt: bentomas: svg suffers from the same dombloat that html suffers from though...it's obviously workable for _most_ applications but at some point you start hitting a bottleneck [21:40] bentomas: true. I guess it really depends on what you are doing. but from all of my web projects, dom isn't the performance bottleneck. It is the loading stuff over the wire. so, I guess I've never had any complaints with html/css/svg. I just need to cache properly [21:41] bentomas: but admittedly, I like pretty simple websites/interfaces/images, so my dom/svg isn't incredibly complicated. [21:42] deanlandolt: bentomas: same here...but some applications just can't operate under that model (think bespin...simple interface, but it doesn't matter how simple your chrome is, if the document you're editing is huge, you'll _never_ be able to work with it using the dom) [21:42] bentomas: good point [21:43] bentomas: speaking of bespin, man I really wish they had gone the Vim route for the editor [21:43] felixge: figured out the main performance bottleneck - creating an object in JS :) [21:43] bentomas: deanlandolt: or did you write it? if so, it is really cool! I just can't let go of vim keybindings [21:43] deanlandolt: how so? [21:44] bryanl has joined the channel [21:44] deanlandolt: bentomas: no, didn't write it...hopin to start hacking on it some day [21:44] deanlandolt: i'm fairly sure keybindings are on the roadmap [21:45] deanlandolt: it's all event driven so you'll be able write plugins that register for given events and do whatever you want...all w/ js [21:49] felixge: damn, Array,push() is slow [21:49] felixge: especially for objects [21:54] inimino: felixge: in v8? [21:54] felixge: inimino: yeah, seems almost 10x slower than setting a object key [21:55] inimino: felixge: btw how are you profiling? [21:55] inimino: heh [21:57] felixge: inimino: 100.000 iterations of adding an object holding a 256 byte str to an array/object [21:57] JoePeck has joined the channel [21:57] inimino: brute force, ok [21:58] inimino: I was wondering if there was some nice profiler for v8 like what Firebug has [21:58] inimino: it's the only thing I ever use Firebug for, but it really is useful to have a profiler... [21:59] felixge: inimino: I think chromium has one [22:00] inimino: ah [22:01] bentomas: so, do we think resolve is a good name for a function on require that returns the location of the file required? [22:01] ericflo has joined the channel [22:01] bentomas: like: require.resolve('sys'); [22:02] bentomas: do we actually think this is useful? It also isn't in the commonjs proposal so... [22:03] inimino: it's useful enough for people writing some kind of require-alike [22:03] ericflo_ has joined the channel [22:04] inimino: doesn't necessarily need to be documented or standardized though [22:08] robrighter has joined the channel [22:12] sudoer has joined the channel [22:18] dnolen has joined the channel [22:20] grantmichaels has joined the channel [22:21] hassox has joined the channel [22:36] felixge: bentomas: making that function would be pretty easy [22:36] felixge: * making it available [22:37] felixge: node.js (the file) has 'findModulePath' which is almost what you want [22:37] felixge: together with a few lines from loadModule [22:37] isaacs: felixge: that'd be nice [22:37] isaacs: while you're at it, talk to kriszyp about his problems with require.paths.push(".") [22:37] felixge: I'll keep it in mind if ryan agrees to refactoring the module loader a bit [22:37] felixge: isaacs: not sure I want to know :) [22:38] isaacs: hahah [22:38] isaacs: basically, in narwhal, you can push "../foo" into the require.paths array, and then it'll resolve to the current module's path. [22:38] bentomas: wow, I really wish I hadn't spent the last 45 minutes writing it then [22:38] isaacs: i hacked up a patch to make that work in node, but it has some weird side effects. [22:38] felixge: bentomas: require.resolve() ? [22:39] isaacs: due to the fact that thigns are cached based on their module ID, not based on the module filename, and that the ID can be a url, etc. [22:39] hassox: morning lads [22:39] felixge: isaacs: yeah, I think that's kind of wrong [22:39] isaacs: so require("./posix") in file.js might fail oddly [22:39] felixge: isaacs: I'll fix that [22:39] voodootikigod has left the channel [22:39] isaacs: felixge: yes, that'd be great. [22:39] ashb: narhwals's module loader has a few oddities to it [22:39] felixge: I actually had quite a few nightmares because of that to debug the module loading code [22:39] isaacs: ACTION nominates felixge as "Lord Of The Require" [22:39] felixge: very hard to see what's going on with the relative ids [22:40] felixge: isaacs: that's a little early, given that I haven't gotten a single patch on that ;) [22:40] ashb: require('./foo') doens't actually look for a foo.js relative to bar.js, but converts it to an abs id and looks for it at the top of the include path [22:40] davidsklar: http://gist.github.com/271611 is a patch against the net2 branch to implement passing file descriptors between processes [22:40] felixge: well, I did fix some require issues after ryan implement commonjs [22:40] felixge: but those were lame [22:40] isaacs: felixge: you're right. maybe ryah will have to remain as LotR-regent until you've come of age. [22:40] felixge: davidsklar: very cool [22:40] ashb: to me relative requite should be explicitly relative to the actual file (which it is in most other commonjs platofmrs) [22:40] felixge: davidsklar: could you make the file format diff? (just name the file .diff) [22:41] felixge: makes it easier to read [22:41] davidsklar: felixge: sure [22:41] felixge: isaacs: yeah. require() is a tough beast, looks trivial on the surface but really isn't [22:41] isaacs: tell me about it! [22:41] davidsklar: felixge: all set [22:42] felixge: davidsklar: much better! [22:42] ashb: felixge: it should be simple [22:42] davidsklar: cool :) [22:42] mikeal has joined the channel [22:43] felixge: davidsklar: very very nice [22:43] isaacs: ashb: that's how it does work, but (in narwhal at least) if you do require("./foo"), it'll first search the current module's path, then the require.paths [22:43] felixge: isaacs: which way is the right ones? narwhal? [22:43] ashb: i think searching in require.paths is a security risk [22:43] davidsklar: felixge: thanks. should be useful for web workers and such [22:43] felixge: ashb: why? [22:43] isaacs: if you do require.paths.push("."), and then do require("foo"), it'll look for "./foo" relative to the current module [22:43] isaacs: felixge: "right" is such a touchy subject... [22:43] ashb: isaacs: what if you do node -I. foo.js [22:44] isaacs: what's -l.? [22:44] ashb: felixge: yeah there is no right here cos narhwal (alone of the 4 other commonjs at the time) had it that way [22:44] ashb: -I. as in add this directory to include path as a cmd line options [22:44] isaacs: oh, ok, didn't know node did that [22:44] ashb: it probably doesn't [22:44] isaacs: hehe [22:44] ashb: but assume it does [22:44] ashb: what would it do? [22:44] isaacs: NODE_PATH=. node foo.js [22:44] ashb: sure [22:45] tav: aside: http://github.com/tav/ampify/blob/master/bin/nodelint.js (for anyone interested in running jslint using node) [22:45] ashb: that behaves differently to require.paths.push('.') ? [22:45] felixge: tav: Very cool! [22:45] tav: ^ also includes file path split/join functions — which would be v.nice to have in the standard library... [22:45] isaacs: ashb: that's something that should be sussed out. not sure if there's a spec or wiki or show of hands or something... [22:45] isaacs: tav: the standard node library has that... [22:45] isaacs: require("path") [22:45] tav: fuck [22:45] ashb: so narwhal is the only one to handl rel paths/mopdules like it does [22:46] tav: isaacs: thanks! [22:46] isaacs: require("path").join("foo", "..", "bar/baz", "./foo") etc [22:46] felixge: tav: is that tuned to not complain about stuff that doesn't apply to node? [22:46] ashb: everythinkg else does it relative to cwd (for paths) or to current file on disk (not to the id in all include paths) [22:46] isaacs: ashb: i see. [22:46] isaacs: that seems... odd for commonjs... [22:47] ashb: in what way? [22:47] isaacs: i mean, isn't there at least an 800-message thread arguing every aspect of the choice in explicit detail? [22:47] tav: i guess http://nodejs.org/api.html is out of date? or does it not include standard node library in general? [22:47] isaacs: complete with doom-and-gloom predictions that commonjs will never succeed unless we all agree? [22:47] tav: (sorry just started with node a few hours ago) [22:47] ashb: isaacs: yeah. but i dont think anyone reelised people weren't thinking the same there [22:47] isaacs: wow. [22:47] ashb: isaacs: i didn't actually read that thread [22:47] isaacs: hahah [22:47] isaacs: ashb: well, that was probably good. [22:48] tav: felixge: it ignores these predef: ['require', 'process', '__filename'] — not sure what else is part of the standard node env... [22:48] isaacs: you probably had laundry to do or something else way more important. [22:48] ashb: but even some of the narwhal guys think that is now the wrong way to behave [22:48] ashb: isaacs: beer to drink [22:48] isaacs: ashb: even better [22:48] bentomas: tav: I just don't think anyone has written docs up for the path module. want to do it? [22:48] felixge: tav: exports, require, module, __filename, process, GLOBAL [22:48] felixge: tav: those will be available in each module on the top level [22:49] ashb: isaacs: so i think you should make '.' behave as it does in a shell (i.e. cwd) and './foo' also (i.e. relative to the actual file only) [22:49] tav: bentomas: sure, assuming the docs are in a sane format... [22:49] tav: felixge: thanks [22:49] felixge: tav: care to make it a standalone project? [22:49] isaacs: ashb: well, in a shell, you CAN put "." into the $PATH [22:49] isaacs: ashb: but it's really super dumb [22:50] tav: felixge: nodelint.js? [22:50] inimino: tav: require is documented [22:50] inimino: tav: as is process, last I looked [22:50] felixge: tav: yeah [22:50] ashb: isaacs: sure, but ut is CWD and doesn't change unless the dir changes [22:50] isaacs: bentomas: tav: actually, i did write up docs for it recently [22:50] tav: felixge: sure [22:50] isaacs: it's in teh latest ry/master [22:50] isaacs: ashb: true that. [22:50] felixge: isaacs: you're a good man [22:50] felixge: I need to write more docs [22:50] bentomas: isaacs: has he just not updated the website? [22:51] ashb: i would be seriously worried that adding '.' to the paths with the behaviour mentioned (being relative to the module where you did it) would confuse people [22:51] felixge: I was gonna document the File class, but ryan seems to not be perfectly happy with it yet [22:51] bentomas: (and by he I mean ryan) [22:51] isaacs: bentomas: the website is up to date with the latest release tag, not necessarily ry/master [22:51] bentomas: ashb: wait, I'm confused, isn't that how it works? [22:52] isaacs: tav: if you want to contribute anything (code, tests, docs), you should pull from github.com/ry/master, make the changes, and send a patch to the mailing list. [22:52] ashb: bentomas: in narwhal perhaps. everywhere else (all other commonjs) '.' is the actualy cwd, not the dir of hte module where you call require.paths.push('.') [22:52] isaacs: ashb: yeah, see, it gets hairy. already, you and i are speaking apples and oranges. [22:52] bentomas: ohhh [22:53] ashb: isaacs: yeah. [22:53] isaacs: foo/bar.js does require.paths.push("."). then baz/quux.js does require("asdf"); it looks for baz/asdf.js, NOT foo/asdf.js [22:53] isaacs: because the "." is resolved relative to the module calling require() [22:53] ashb: which to me coming form any other language is just confusing [22:53] ashb: since i'd epxected it to look for just plain asdf.js in the $cwd [22:55] isaacs: well, having a require.paths to search is fine to me. that's a lot like the way that $PATH works for unqualified executables in the shell. [22:56] isaacs: if I type "bash", it looks for the bash in my $PATH, not in . [22:56] ashb: but if oyu type ./bash its doesn't saerch :) [22:56] isaacs: ashb: exactly [22:56] isaacs: if I do PATH=., then it'll always look for ./bash when i do "bash", but that's dangerous and dumb. [22:58] tav: isaacs: thanks [22:59] isaacs: tav: no problem! there's a lot to do, so if you feel like helping, then that's great! [22:59] isaacs: having jslint in node is neat. [23:00] ashb: isaacs: yes, adding . to your PATH is dumb, but how dumbd it is isn't the issue [23:00] ashb: its how much of a supprise that '.' doens't mean the same as '.' every where else [23:01] isaacs: ashb: right. if i do PATH=., and then run a script at ./foo/bar.sh, and it calls "baz.sh", then it'll look for ./baz.sh, NOT ./foo/baz.sh [23:01] ashb: and thats up to you/ry etc to decide. i think its quite widly different [23:02] isaacs: i'm perfectly happy saying "you can't add . to your require.paths, because that's as bright as giving a 2-year-old a chainsaw" [23:02] ashb: er, that being wildly different is wrong [23:02] isaacs: but i'm predicting the response of "but this is javascript, we do dangerous dumb stuff all the time, and besides, my toddler really LIKES chainsaws." [23:02] isaacs: and i have no good response to that. [23:03] ashb: :) [23:03] inimino: if "../foo" is going to be allowed, why would "." not? [23:03] inimino: and aren't the issues the same? [23:03] isaacs: inimino: sure. [23:03] isaacs: but "." is less typing [23:03] isaacs: (for teh purpose of discussion) [23:03] ashb: ../foo and . are different things [23:03] ashb: well, i guess they might not be [23:03] tav: 2 questions: a) are there other "builtin" modules like path that one should know about? i looked in lib/node/libraries — but obviously there are more libraries... [23:03] ashb: but they are different types of paths [23:03] isaacs: the question is whether you should be able to put relative paths in your require.paths and how they'd resolve. [23:04] tav: b) has anyone ported env.js to node yet? [23:04] inimino: they're both relative paths [23:06] hassox: tav: what is env.js? [23:06] isaacs: tav: a) ls -l /usr/local/lib/node_libraries [23:06] isaacs: tav: b) not that i'm aware of. [23:07] tav: hassox: http://ejohn.org/blog/bringing-the-browser-to-the-server/ (a rhino implementation of a browser-like environment so you can run jQuery, etc.) [23:07] hassox: gotcha [23:07] isaacs: tav: you did jslint, i think you've got the stuff to do it! [23:07] isaacs: go for it! [23:07] isaacs: tav: http://github.com/tav/node-env has your name written all over it! [23:08] tav: lol [23:08] isaacs: (well, not all over it, but it's got your name in the path at least...)( [23:09] tav: isaacs: http://gist.github.com/271701 is my output for ls lib/... — and from what i can tell 'path' is an internal module (?) and not one of the standard library modules? [23:09] isaacs: oh, yeah, there are other things in node.js [23:10] isaacs: sadly, if anything isn't documented, you have to rtfs to learn about it [23:10] isaacs: i think in ry/head, most of the reasonably-stable stuff is documented. [23:10] isaacs: ry/master, that is [23:11] inimino: the documentation is pretty up-to-date, especially with the stable release [23:11] isaacs: inimino: it's up to date, but woefully incomplete [23:11] isaacs: require("path") is a little bit important. [23:12] inimino: well, yes, I suppose so [23:12] inimino: anyway, bbiab [23:13] tav: ah, posix and path are the only special "internal" modules it seems [23:14] tav: i guess it makes sense — node.js needs that functionality to load modules [23:15] isaacs: tav: yeah [23:15] isaacs: basically just the bootstrapping for require() [23:16] tav: thanks isaacs [23:16] isaacs: tav: np, have fun [23:23] hassox: anyone familiar with the "Passing FDs between processes" thread on the ML? [23:24] xantus: no, but I understand the technique [23:25] tav: hmz, not on the mailing list, but you can use sendmsg to pass them around [23:25] isaacs: hassox: i've been watching with interest. [23:25] isaacs: it seems interesting. [23:25] felixge: ok I must be dreaming this up, but I think dirty can iterate over 33 mio. documents / sec ( http://github.com/felixge/node-dirty/commit/f8fc7698cdd401c868d9e2a98bb47ae3f0d82901 ) [23:26] tav: felixge: not happy with redis ? [23:26] hassox: isaacs: is that something that could be used to pass the actual connection object about? [23:26] felixge: tav: never used it [23:26] felixge: tav: I want something I can use with node.js [23:27] tav: it should take about an hour to write a fairly comprehensive redis client — if there isn't one already... [23:27] felixge: I also want something very bare-bone. No network server, just direct shared memory that can easily persist to disk [23:27] tav: http://github.com/fictorial/redis-node-client [23:28] felixge: tav: yeah, but why? I think I can easily make something as fast as reddis in node [23:28] felixge: (as in 98 lines of code easy) [23:28] felixge: the huge advantage & drawback at the same time being that it only works in node.js [23:28] felixge: but I'm adressing this with the file format, its simply newline terminated JSON [23:29] isaacs: if you can't do it in 100 lines javascript, you probably don't need to do it at all. [23:29] felixge: hehe [23:29] kriszyp: felixge: I've been wanting to build something like this as well, thats cool that you have already got something going. [23:29] kriszyp: I wanted to run an idea by you with that... [23:29] felixge: kriszyp: sure, go ahead :) [23:30] tav: hmz, redis has direct access to malloc and friends and it's really well coded... so not sure how one could compete with it.... but, still, k/v stores are good! =) [23:30] felixge: tav: reddis has network I/O overhead and parses a protcol [23:30] felixge: that's actually not quite trivial [23:30] felixge: in terms of overhead [23:31] felixge: tav: their benchmark page lists the 'record' for GET as 81000 docs / sec [23:31] kriszyp: I've been working on a persistence/store framework (http://github.com/kriszyp/perstore/) , I want to be able to plug in various differnt stores to a common REST handler. So I've been trying to roughly follow the W3C database api spec for object stores: http://dev.w3.org/2006/webapi/WebSimpleDB/#object-store-sync (but naturally using promises for async) [23:32] felixge: and that's with multiple clients accessing it in parallel [23:32] kriszyp: anyway, wanted to see if the API looked reasonable to you [23:32] kriszyp: for something like node-dirty... of course I could write an adapter for it [23:33] kriszyp: get, put, delete, and create are the basic methods [23:33] felixge: kriszyp: yeah, my idea would be to ship adapters with it [23:33] kriszyp: I see [23:33] felixge: but you should be able to just use the k/v store without any bells in whistles by default [23:33] kriszyp: i think it is actually already pretty close, from what i can tell [23:34] felixge: looking at your links now [23:34] rtomayko has joined the channel [23:34] isaacs: kriszyp: so, about that whole require.paths.push("../foo") thing... [23:34] felixge: kriszyp: I've been thinking about a Model layer for dirty as well, but I bailed out because that is actually hard ;) [23:34] kriszyp: oh yeah [23:34] isaacs: you might want to check the logs. we were discussing that a big. [23:34] isaacs: *bit [23:34] kriszyp: well, you could have one for free with perstore :) [23:35] tav: felixge: interesting — so it's more of a PersistentObject type? [23:35] felixge: tav: It's pretty much if Redis and CouchDB had a child and put it right into your process [23:35] tav: hehe [23:35] tav: nice [23:35] kriszyp: probably one of the more unique ideas is the use of faceting (for ocap style security) [23:36] felixge: tav: you get CouchDB like views (which you have to persist those yourself, but its easy) and Redis memory-speed goodness [23:36] felixge: the main limitation is that you're fucked once your db size > RAM size [23:36] felixge: kriszyp: lets work on making that happening [23:36] felixge: I'm all for it [23:36] kriszyp: cool [23:36] isaacs: felixge: it'd be nice to have it look at the memory usage, and start caching to disk (or dropping items) once you get above a certain configurable limit. [23:36] kriszyp: the project is definitely still in state where I am open to suggestions for direction [23:37] felixge: kriszyp: this is all there is so far: http://github.com/felixge/node-dirty/blob/master/lib/dirty.js [23:37] kriszyp: but it already is functional with JSON schema validation on models, and faceting and so forth [23:37] felixge: kriszyp: what else do you need? [23:37] felixge: nice [23:37] kriszyp: right, thats what I was looking at [23:37] kriszyp: (your dirty.js module) [23:37] felixge: isaacs: memory usage is probably going to be somewhat horrible compared to redis [23:37] isaacs: felixge: well, yeah, that's the point, right? [23:37] felixge: isaacs: yeah [23:37] isaacs: but at least, it could keep itself in check somewhat [23:38] tav: the main super nice thing from redis btw is it's support for a list primitive — which supports atomic push/pop, and recently added blocking push/pop — which is great for storing/getting comet data [23:38] felixge: isaacs: yeah, but as soon as you have too many records, you simply can't filter through them efficiently anymore which is the whole point. If you hit that limit you'll have to migrate to something like CouchDb [23:38] felixge: (or shard to multiple machines) [23:38] tav: ^ be good to have in dirty =) [23:39] felixge: tav: dirty has it [23:39] kriszyp: another thing that i use that might be useful at some point for dirty is something like http://github.com/kriszyp/commonjs-utils/blob/master/lib/json-ext.js so you can natively persist dates and some other types that aren't handled by pure json [23:39] isaacs: felixge: this is a problem that's tackled really interestingly by yahoo's YTS server [23:39] isaacs: YTS is a forward-proxy caching type thing that it uses for edge caches. [23:39] tav: felixge: oooh! [23:39] felixge: kriszyp: probably not as an adapter [23:39] isaacs: basically, when the memory hits a certain level, it drops a set percentage of the records in memory. it turns out that keeping track of the timestamps is not worth the cost. [23:40] kriszyp: oh, it is just an JSON parser/stringifier that does more than the native one [23:40] isaacs: so it just drops them randomly [23:40] felixge: isaacs: ah, yeah that could totally be done [23:40] isaacs: and, since it's just a forward proxy, if they're important, they'll get fetched again soon anyhow [23:40] ashb: kriszyp: mongodb's BSON might be of interest [23:40] ashb: (binary JSON with regexp, date and blob support) [23:41] kriszyp: is it human readable? [23:41] mikeal: i really don't like BSON [23:41] tav: BSON-- [23:41] bryanl has joined the channel [23:41] felixge: isaacs: it should be pretty easy to do this with dirty. that's a very cool idea [23:41] kriszyp: does it properly superset JSON and subset JS? [23:41] mikeal: BSON is not human readable [23:41] mikeal: it's Binary, that's what the B stands for :) [23:42] isaacs: felixge: oh, also, since you've got the ability to cache to disk, you could make the memory store just be a proxy for the disk store. [23:42] felixge: kriszyp: yeah, but I don't want slow serialization. I hope v8 has the JSON stuff reasonably optimized [23:42] kriszyp: right, json-ext just will serialize dates in objects as something that is easily eval'able: {someDate: new Date(34343434), otherProp: 4, ...} [23:42] kriszyp: well, it uses the native serializer, but it uses the custom serialization function parameter, I think [23:43] mikeal: kriszyp: but that breaks most other serializers [23:43] kriszyp: I don't know how much that slows things down... [23:43] felixge: kriszyp: actually it would only slow down the disk writes, not the memory store itself [23:43] kriszyp: well, if you are writing the parser and the serializer [23:43] kriszyp: right [23:43] felixge: kriszyp: well, the memory store will suffer indirectly by the event loop being held up [23:43] kriszyp: right [23:43] kriszyp: what do you do with dates now? just let them be lossy? [23:44] felixge: kriszyp: whatever JSON.stringify() does to them in v8 [23:44] kriszyp: it's lossy [23:44] felixge: ok [23:44] felixge: then people should store their dates as strings [23:44] kriszyp: just turns them into a string representation [23:44] felixge: why is it lossy in v8? [23:44] kriszyp: heh [23:44] kriszyp: when you parse it is not the same thing as when you serialized it [23:44] kriszyp: it is a string instead of a date [23:44] mikeal: people should store dates and ISO 8601 formated [23:44] kriszyp: and I've noticed that people use dates ;) [23:44] mikeal: and/as [23:45] felixge: isaacs: I think I will add support for Dirty databases that are only kept on disk, but can partially be streamed into memory when needed [23:45] mikeal: what can you really do with a native js date anyway? [23:45] mikeal: native js dates suck…. badly [23:45] felixge: isaacs: so you get really slow mass-storage and very fast memory storage. [23:45] mikeal: that's why we have date.js :) [23:45] isaacs: felixge: that seems to be the best way to do it, because memory is so flaky anyhow. [23:46] isaacs: felixge: also, then you can have a "refresh" signal that just drops the in-memory store. [23:46] felixge: kriszyp: yeah, well I would like to push issues like this into a higher layer. Like perstore [23:46] deanlandolt: mikeal: i believe es5 adds ISO helpers [23:46] kriszyp: yeah, thats fine [23:46] kriszyp: just throwin that out there for you to think about [23:46] isaacs: felixge: watch out, though, going down this road. before you know it, you'll have indexes, hashes, etc. [23:46] felixge: isaacs: I'll have to think about it a bit more, I talked myself out of it the other night [23:46] kriszyp: yeah, are you going to add indexes? [23:46] felixge: kriszyp: no, that's too specialized [23:46] kriszyp: O(log n) access is nice :) [23:46] mikeal: deanlandolt: that helps, but it doesn't provide the support necessary for doing timezones, which is necessary for non-trivial datetime handling [23:47] felixge: I don't want to optimize disk usage really [23:47] felixge: other than the write new documents or read whole db case [23:47] deanlandolt: mikeal: point taken [23:47] deanlandolt: felixge: what happens if you can't read the whole db into memory? [23:47] felixge: It's honestly not my area of expertise, so I would need guidance from somebody more experienced if I did that [23:47] felixge: deanlandolt: you know its time to shard [23:48] kriszyp: indexes are much simpler if they remain in memory, thats for sure [23:48] deanlandolt: fair enough [23:48] felixge: I want to make the current implementation a bit more faster for now and call it 1.0 [23:48] kriszyp: indexes get complex when you start dealing with disk-level access and optimized b-trees. But for an in-memory store, I bet there is a js red-black tree impl out there somewher that would make it easy [23:49] felixge: Mainly I'd like to get closer to 1 mio writes/sec [23:49] felixge: kriszyp: I found a nice JS B+ Tree implementation [23:49] kriszyp: cool [23:49] deanlandolt: felixge: link? [23:49] felixge: http://blog.conquex.com/?p=84 [23:50] tav: hmz, avoid B+ trees [23:50] felixge: at least it looks reasonable to me, knowing little more about it other than that its optimized for disk access [23:50] felixge: I'm wondering if I should care about spinning disks at all [23:50] felixge: and focus on solid state instead [23:51] kriszyp: if you are just reading in the whole db into memory, I wouldn't worry about disk-based indexes [23:51] kriszyp: could just keep it in memory [23:51] tav: with disk access — it's key (for performance) to minimise seeks [23:51] felixge: kriszyp: yeah, I kind of like the simple file format right now [23:51] kriszyp: and it looks like that B+ tree would do the in-memory index quite well [23:51] felixge: tav: yeah, but that's not (as?) true with solid state [23:52] felixge: kriszyp: why? Doesn't the benchmark say its slower than using a JS object directly? [23:52] felixge: kriszyp: only range search looks good [23:52] kriszyp: I don't understand [23:53] felixge: oh, I guess that it's expected. You get slower writes in exchange for faster seeks? [23:53] tav: sure: random access on ssds is far more performant — but how many of your users are going to be using them? [23:53] deanlandolt: tav: more by the month [23:54] deanlandolt: felixge: what do you mean only range search looks good? [23:54] felixge: deanlandolt: the link I pasted, it has some benchmark numbers posted [23:54] deanlandolt: you mean because it takes so long to populate vs. an object? [23:54] mikeal: CouchDB's btree does big sequential writes to an append only file, so it's only effected by the seek time performance issues to the extent that other writers on the OS might be seeking to other areas of the disc [23:55] felixge: mikeal: big sequantial writes? for the views, but not for storing new docs, right? [23:56] felixge: afaik couch doesn't buffer new docs in memory to flush them to disk in bundles [23:56] felixge: (by design) [23:56] mikeal: when you update a section of the btree it doesn't seek to that area on disc and overwrite it, it's an append only file so it just writes to references and writes the new section to the append only file [23:56] mikeal: so it's not seeking around on disc really [23:56] felixge: ah, we're confusing each other here :) [23:57] tav: aside: some nice alogirthms that'd be nice to see — in dirty or otherwise — http://blog.notdot.net/2009/11/Damn-Cool-Algorithms-Spatial-indexing-with-Quadtrees-and-Hilbert-Curves [23:57] tav: ^ i'd also add skip lists to that... [23:57] mikeal: there's also a btree for docs [23:57] mikeal: because documents have revisions [23:57] felixge: mikeal: kriszyp said "and it looks like that B+ tree would do the in-memory index quite well" [23:57] felixge: which I am confused about [23:57] felixge: since IMHO there is no reason to use a b+ tree in memory [23:58] mikeal: i don't have indexes that are small enough to keep in memory [23:58] mikeal: felixge: it gets you out of locking [23:58] felixge: tav: looks interesting [23:58] mikeal: because you can use the current state of the btree for any read requests, and when you write you do it append only and don't update the reference to the new data until you finish the write [23:59] mikeal: if you don't use append only then you'll need to lock any reads of the btree while you're doing your write because you could attempt to read in an inconsistent state