[00:00] CIA-77: node: 03Micheil Smith 07master * r57ea07a 10/ (lib/http.js lib/net.js src/node.cc lib/freelist.js): Moving the http.js, net.js FreeList to being standalone. - http://bit.ly/9a7pvi [00:01] devinus has joined the channel [00:02] devinus: http://www.reddit.com/r/programming/comments/bq0tf/fotofoo_an_image_management_solution_using/ [00:04] _ry: devinus: you should put up an online demo [00:04] devinus: _ry: i don't have access to a boxen :[ [00:04] devinus: _ry: my blog is on tumblr [00:05] isaacs: devinus: srsly? [00:05] devinus: isaacs: ya.... [00:06] isaacs: devinus: this looks pretty neat. i might have to install it on my box. [00:07] isaacs: any bets on how long before it's nothing but goatse? [00:08] devinus: isaacs: pretty soon prob. it's totally not safe to run under any privileged user, btw [00:08] devinus: gotsta lock it down [00:08] isaacs: devinus: ok. [00:08] isaacs: seriously, i don't know when i'll get around to setting it up. but i bookmarked it in my "procrastination fodder" folder. [00:09] _ry: would be cool if web browsers could connect to unix sockets [00:09] devinus: awesome [00:19] quirkey has joined the channel [00:23] mikeal: anyone know how to save a file with a filter applied in wireshark [00:28] joshbuddy has joined the channel [00:28] joshbuddy has joined the channel [00:29] mattly has joined the channel [00:34] mikeal: _ry: I've got an HTTP dump for yah [00:34] mikeal: nothing special tho [00:35] JimBastard_ has joined the channel [00:35] _ry: mikeal: ok [00:35] JimBastard_: does anyone know if there are any tools for parsing a .gitsubmodule file? [00:35] JimBastard_: ideally i'd want .gitsubmodule in, JSON out [00:36] mikeal: _ry: also, it's not always failing in the same place [00:36] mikeal: the same place in the response that is [00:36] mikeal: always the same place in code :) [00:36] _ry: JimBastard_: use the ini parser [00:37] _ry: JimBastard_: require('ini').parse(data) [00:37] JimBastard_: ohh shit thats built in? [00:37] _ry: yeah [00:37] JimBastard_: ill try it out thanks for the tip [00:37] confounds has left the channel [00:38] mikeal: the total response size is ~ 697470 [00:38] _ry: mikeal: only one response though? [00:38] _ry: only one request? [00:38] mikeal: yup [00:39] mikeal: it always fails on the same request, just at a different point in the request [00:40] JimBastard_: yeah that looks pretty good _ry, now i can autogenerate some nice docs for my submodules (i think im using like 8 node projects right now, maybe more) [00:43] _ry: JimBastard_: require('nice_docs').autogen(data) [00:43] JimBastard_: ? [00:43] _ry: joke. nm :) [00:43] JimBastard_: :p [00:44] JimBastard_: hook.io will have continuous documentation as soon as michael jackson updates his shit to 1.9 [00:44] _ry: 0.1.90 [00:44] JimBastard_: aye [00:45] JimBastard_: the documentation re-generates on every compile (for now). so changes show up immediately in the docs [00:45] JimBastard_: even the README.md on github is autogenerated [00:45] JimBastard_: http://github.com/marak/hook.io [00:50] kenneth_reitz has joined the channel [00:52] steadicat has joined the channel [00:53] joshbuddy has joined the channel [00:53] joshbuddy has joined the channel [00:57] jashkenas has joined the channel [00:58] chewbranca: anyone have any experience with graphics acceleration with kvm? [00:58] devinus has joined the channel [00:59] jashkenas has joined the channel [01:07] gwoo: JimBastard: what is an actioner? [01:07] JimBastard_: its the action dispatcher [01:08] JimBastard_: it dispatches webhook actions [01:08] JimBastard_: im still working on the webhook book should rough draft by jsconf [01:08] JimBastard_: i also wanna rename some of the internals [01:09] gwoo: are you considering changing that to "dispatcher"? [01:09] JimBastard_: there are two dispatchers [01:09] gwoo: oh [01:09] JimBastard_: we could merge them both into one, but for now its much easier [01:09] JimBastard_: the listenerer and the actioner [01:09] gwoo: what's the other one? [01:09] JimBastard_: listenerer is currently called "hooker" [01:09] JimBastard_: that is gonna get changed, there is an issue open [01:09] gwoo: ah ok [01:10] JimBastard_: im not set in stone about either of those terms [01:10] JimBastard_: you reading through the code? [01:10] gwoo: yeah [01:10] JimBastard_: cool man [01:10] JimBastard_: let me know if you have any questions or comments [01:10] jashkenas has left the channel [01:10] malkomalko has joined the channel [01:10] gwoo: i really like the idea [01:11] gwoo: there is so much potential [01:11] JimBastard_: im working on http://github.com/marak/hook.io/issues#issue/17 for two more days [01:11] JimBastard_: ya ya [01:11] JimBastard_: everything is uber simple too [01:12] JimBastard_: i could definitely use help [01:12] gwoo: im gonna get everything running [01:12] JimBastard_: hee hee good luck [01:12] gwoo: then i would be psyched to be part of the front end team [01:13] JimBastard_: yeah the front end is slapped together with jquery fu [01:13] gwoo: ha [01:13] JimBastard_: ive been working on fleshing out hookIO.js which is the browser-side JavaScript API client [01:13] JimBastard_: it has a JSON rpc wrapper and jquery.ajax i think [01:13] gwoo: item 2 [01:13] JimBastard_: so anyone who wants to build a front-end or a widget can just piggy back off that file [01:14] gwoo: nice [01:15] JimBastard_: gwoo its possible you might get some pathing errors while doing install. maybe not... i've been adding a shit ton of stuff in the past few days [01:15] JimBastard_: if all goes well the install should only take like 20 seconds though [01:15] gwoo: ok lets see about that [01:15] gwoo: :) [01:20] gwoo: my interwebs are slow today [01:20] JimBastard_: im pushing to the repo like every 20 minutes [01:20] gwoo: multipart [01:21] JimBastard_: ? [01:21] gwoo: looks like node_debug might require it [01:22] devinus has joined the channel [01:22] gwoo: Cannot find module 'multipart' [01:22] _ry: gwoo: done [01:22] _ry: gone [01:22] gwoo: yup [01:23] JimBastard_: gwoo is this in hookIO? [01:23] gwoo: JimBastard_: yeah [01:23] JimBastard_: uhhh [01:23] brapse has joined the channel [01:23] JimBastard_: what version of node you running [01:23] gwoo: 1.9 [01:23] gwoo: 0.1.9 [01:23] JimBastard_: http://github.com/Marak/hook.io/blob/master/hookio/lib/multipart.js [01:24] JimBastard_: where is it bombing out [01:24] JimBastard_: the file is there, maybe its not being linked up [01:24] gwoo: fu.js i think [01:24] JimBastard_: node_debug? [01:25] JimBastard_: yeah it must be [01:25] gwoo: yup [01:25] JimBastard_: one sec [01:25] JimBastard_: im slightly confused about how to manage submodule updates, trying something [01:26] JimBastard_: gwoo: can you check your fu.js file for this line multipart = require( './multipart' ), [01:26] gwoo: cd lib/node_debug && git pull [01:26] JimBastard_: and then see if multipart.js is there [01:27] gwoo: JimBastard_: for me it was ../../multipart [01:27] JimBastard_: multipart.js exists in two places [01:27] gwoo: ah [01:27] JimBastard_: hrmmm [01:27] JimBastard_: thats really strange [01:27] gwoo: well it's running for me [01:27] gwoo: deprecation warning: process.mixin will be removed from node-core future releases. [01:28] JimBastard_: yeah [01:28] gwoo: The 'tcp' module is now called 'net'. Otherwise it should have a similar interface. [01:28] JimBastard_: gotta fix that stuff [01:28] JimBastard_: havent gotten around to it yet [01:28] gwoo: yup [01:28] JimBastard_: did it autogen your docs? [01:28] gwoo: up [01:28] gwoo: yup [01:28] JimBastard_: cool beans, im gonna get Mint integration asap [01:28] JimBastard_: so you probaly want a front-end right [01:30] gwoo: JimBastard_: is there a #hook.io? [01:30] bmizerany has joined the channel [01:31] bmizerany has joined the channel [01:31] JimBastard_: nah there is no need. 90% of hook.io is existing node modules [01:31] JimBastard_: if you dont want to derail chan you can pm me [01:36] dnolen has joined the channel [01:38] _ry: JimBastard_: hook.io isn't on library_compatibility [01:38] JimBastard_: ? [01:39] _ry: http://wiki.github.com/ry/node/library-compatibility [01:39] JimBastard_: what do i have to do? [01:39] _ry: link to a specific commit [01:39] _ry: which works with 0.1.90 [01:39] _ry: for historical reference [01:40] _ry: or tag a version [01:40] JimBastard_: yeah [01:40] JimBastard_: i mean im still 0.0 [01:40] JimBastard_: ill link all that stuff up soon [01:40] _ry: ok [01:40] charlesjolley has joined the channel [01:42] RayMorgan has joined the channel [01:48] malkomalko has joined the channel [01:51] tmpvar has joined the channel [01:52] kriskowal_ has joined the channel [01:53] RayMorgan has joined the channel [01:55] mjijackson has joined the channel [01:57] kriskowal has joined the channel [02:00] JimBastard_: you still messing around gwoo ? [02:01] creationix: Oops, it appears I pushed an unfinished article to howtonode.org [02:01] creationix: http://howtonode.org/hello-node#comment-44536459 [02:04] jeffreyt has joined the channel [02:06] pavelz has joined the channel [02:08] _ry: haha [02:08] creationix: cleaned up the ending [02:09] _ry: nginx's mem usage is http://howtonode.org/hello-node#comment-44536459 [02:09] _ry: oops [02:09] _ry: ! [02:09] creationix: nginx? [02:09] _ry: nginx's mem usage is ridiculous [02:09] creationix: ahh [02:10] _ry: (i had to look up ridiculous in the dictinary *_*) [02:10] creationix: _ry: I'm thinking about using varnish for the new howtonode.org so it will scale well [02:10] creationix: the beta site is running on it now with about 3000 reqs/sec [02:10] _ry: it's like - less that 2mb rss at 500 concurrent users [02:11] _ry: s/that/than/ [02:11] creationix: and people say threads are the only way to scale [02:11] JimBastard__ has joined the channel [02:11] JimBastard__: i seem to be multiplying today [02:11] JimBastard__: i cant wait till i get the irc protocol working right for hook.io, im sure ill get banned from freenode again [02:12] _ry: node is pretty much on par though in terms of speed [02:12] _ry: just takes about 20 times the memory :) [02:12] creationix: _ry: I wonder if there is a way to speed up subprocesses [02:12] creationix: that's what is killing the howtonode engine [02:12] _ry: can't make subprocesses for each request [02:13] _ry: that's what we learned from cgi [02:13] _ry: process creation simply isn't fast enough [02:13] creationix: well, I don't think git allows a single process to make multiple show commands, or does it? [02:13] creationix: I'm using a bare git repo as a file repository. I query it using git show commands [02:14] _ry: you have to cache [02:14] creationix: well, yeah, caching works, that's basically why I'm using varnish [02:14] rtomayko: creationix: some git commands take an arg that lets you pipe stuff in on stdin and then read from stdout. so you could have a long-lived process that way. [02:14] rtomayko: not sure about git-show though [02:15] creationix: rtomayko: I couldn't find anything when I looked, but I didn't look too long [02:15] creationix: if course caching makes perfect sense in this case. queries with specified sha hashes are never going to change [02:15] rtomayko: we run lots of short-lived git processes at github. but we cache the piss out of everything too. [02:15] rtomayko: exactly. that's the secret to everything. [02:16] creationix: and I can manually invalidate any queries against head whenever there is a new commit [02:16] creationix: rtomayko: that's why popular projects load faster [02:16] rtomayko: or just resolve HEAD to a SHA1, since that's a simple file read [02:16] creationix: where is that stored? [02:17] rtomayko: .git/HEAD [02:17] rtomayko: .git/refs/heads/* [02:17] creationix: ahh, that's easy :) [02:17] creationix: I think that's a good idea [02:17] rtomayko: one cool thing about git -- this is debatable I guess, some people hate it -- is that the file hierarchy is supposed to be like a stable API [02:17] rtomayko: almost like kernel /proc [02:17] rtomayko: so the shit under .git is fair play. fuck a library [02:18] creationix: that's good for people scraping it [02:18] creationix: ok, so I can do a quick fs.readFileSync to refs/heads/master and use that as the sha [02:18] rtomayko: boom [02:18] creationix: or even an async readFile, not sure which would be faster [02:19] creationix: both are worlds faster than loading up git in a subprocess [02:19] bmizerany has joined the channel [02:20] creationix: _ry: Do you think it's safe to readFileSync on a small file that I'm going to read at virtually every request? [02:20] creationix: I imagine the OS will cache the lookup [02:20] _ry: i guess - i won't do that [02:20] creationix: ok, I guess an async read isn't too much overhead [02:20] creationix: and it doesn't block in case there is trouble [02:20] _ry: i don't think it's much overhead [02:21] stevestmartin has joined the channel [02:21] _ry: but you'd be better off just reading it every 5 seconds [02:21] _ry: probably [02:21] creationix: _ry, or watch the file [02:21] _ry: don't watch files [02:21] _ry: it's expensive [02:21] _ry: :) [02:22] _ry: i'm the wrong person to talk to about this [02:22] _ry: i think things should be served from memory always [02:22] Aria has joined the channel [02:23] creationix: _ry: so, I've got about 300mb of free ram on my host [02:23] creationix: should I just cache everything possible and serve the site from node directly [02:23] creationix: instead of using varnish or some elaborate caching to static files system using nginx as a proxy [02:24] JimBastard_ has joined the channel [02:24] _ry: varnish sounds good [02:25] _ry: gtg [02:25] _ry: later [02:26] gwoo: later [02:26] gwoo: ACTION had a long phone call [02:27] mjijackson has joined the channel [02:30] admc has joined the channel [02:31] JimBastard_: so gwoo did you get it running? [02:31] gwoo: yup [02:31] JimBastard_: any front-end? [02:31] gwoo: not yet [02:31] JimBastard_: so you can clone the one i use @ hook.io [02:31] towski has joined the channel [02:32] gwoo: yeah i did that [02:32] JimBastard_: and if you use nginx you can copy the nginx.conf [02:32] gwoo: just getting it setup [02:32] JimBastard_: you will need to reverse proxy the requests [02:32] JimBastard_: so /api points to :8000/jsonrpc [02:32] gwoo: ah ok [02:32] JimBastard_: the nginx.conf has it all [02:32] JimBastard_: should be the root of the front-end site [02:34] gwoo: yeah i was just looking at that [02:35] JimBastard_: also i would pull again [02:35] JimBastard_: im updating a lot [02:35] JimBastard_: if you know any nginx fu i think the routes could be defined with one wildcard [02:40] gwoo: yup should not be too hard. i'll take a whack at it [02:41] RayMorgan has joined the channel [02:48] bmizeran_ has joined the channel [02:48] pedro has joined the channel [03:03] creationix: rtomayko: you still there? [03:04] creationix: I can't seem to find a way to get the sha1 for HEAD in a bare repo [03:04] creationix: my refs folder is empty [03:07] devinus has joined the channel [03:19] kenneth_reitz has joined the channel [03:24] kenneth_reitz has joined the channel [03:26] gwoo: creationix: you need to make a commit [03:27] joshbuddy has joined the channel [03:27] creationix: I can't make commits in a bare repo, there are no files to commit [03:27] creationix: I think the bare-refs file has what I need though [03:27] creationix: I mean packed-refs [03:28] BinaryPie has joined the channel [03:39] mikeal has joined the channel [03:48] creationix: oh yeah, 2.6 million lookups/second [03:48] creationix: that works [03:49] creationix: way faster than opening a child process [03:49] gwoo: thats a bit faster than before ;) [03:50] PyroPete1 has joined the channel [03:50] creationix: it does two fs.readFiles in parallel, calculates the sha based on their contents and then caches the result for 10 ms [03:50] creationix: I'll probably do something more reasonable like a 5 second cache [03:52] aho: or LRU [03:53] creationix: well, the 2.6 million kinda cheats, since I was issuing the request in sync time, they were all cached, the timeout to invalidate my cache never fired [03:53] creationix: using process.nextTick on every call I only get 62,000/sec [03:54] creationix: still not bad [03:56] mjijackson has joined the channel [03:56] bmizerany has joined the channel [04:19] dgathright has joined the channel [04:19] RayMorgan has joined the channel [04:20] cedricv has joined the channel [04:30] mcarter has joined the channel [04:39] mikeal has joined the channel [04:46] jpoz has joined the channel [05:00] Aria has joined the channel [05:00] brapse has joined the channel [05:02] codehero has joined the channel [05:04] bmizerany has joined the channel [05:05] mikeal has joined the channel [05:09] dekroning has joined the channel [05:18] binary42 has joined the channel [05:26] mikeal has joined the channel [05:37] charlesjolley has joined the channel [05:37] dekroning has joined the channel [05:37] mjijackson has joined the channel [05:37] pavelz has joined the channel [05:37] hassox has joined the channel [05:37] creationix has joined the channel [05:37] sh1mmer has joined the channel [05:37] Iveaux has joined the channel [05:37] frodenius has joined the channel [05:37] CodeOfficer has joined the channel [05:37] gwoo has joined the channel [05:37] fictorial has joined the channel [05:37] kjeldahl has joined the channel [05:37] tk has joined the channel [05:37] jbrantly has joined the channel [05:37] freshtonic has joined the channel [05:37] bengl has joined the channel [05:37] brainproxy has joined the channel [05:37] KungFuHamster has joined the channel [05:37] philippbosch has joined the channel [05:37] Atmoz has joined the channel [05:37] avidal has joined the channel [05:37] stalled has joined the channel [05:37] ryan[WIN] has joined the channel [05:37] deanlandolt has joined the channel [05:37] polyrhythmic has joined the channel [05:37] mde has joined the channel [05:37] idoru has joined the channel [05:37] kloeri has joined the channel [05:37] Sembiance has joined the channel [05:37] mortens has joined the channel [05:37] konobi has joined the channel [05:37] _ry has joined the channel [05:37] elbart has joined the channel [05:37] jacobat_ has joined the channel [05:37] jackyyll has joined the channel [05:37] leifkb has joined the channel [05:37] Connorhd has joined the channel [05:37] inarru has joined the channel [05:37] beppu has joined the channel [05:37] nodejs_v8 has joined the channel [05:37] halorgium has joined the channel [05:38] bmizerany has joined the channel [05:38] jpoz has joined the channel [05:38] kenneth_reitz has joined the channel [05:38] admc has joined the channel [05:38] tmpvar has joined the channel [05:38] mattly has joined the channel [05:38] darkf has joined the channel [05:38] sveisvei has joined the channel [05:38] aho has joined the channel [05:38] xer0x|away has joined the channel [05:38] Gruni has joined the channel [05:38] cainus_ has joined the channel [05:38] trochala has joined the channel [05:38] gbot2 has joined the channel [05:38] inimino has joined the channel [05:38] blazzy has joined the channel [05:38] mrd` has joined the channel [05:38] micheil has joined the channel [05:38] bajeczka has joined the channel [05:38] ithinkihaveacat has joined the channel [05:38] nefD has joined the channel [05:38] siculars has joined the channel [05:38] jan____ has joined the channel [05:38] mediacoder has joined the channel [05:38] emyller has joined the channel [05:38] marienz has joined the channel [05:38] Elfix has joined the channel [05:38] tav has joined the channel [05:38] JshWright has joined the channel [05:38] webben has joined the channel [05:38] tlrobinson has joined the channel [05:38] ddollar has joined the channel [05:38] NickP_ has joined the channel [05:38] onar has joined the channel [05:38] hober has joined the channel [05:38] evilhackerdude has joined the channel [05:38] ashb has joined the channel [05:38] tlockney has joined the channel [05:38] rudebwoy has joined the channel [05:38] dmpk2k has joined the channel [05:45] okito has joined the channel [05:47] okito has joined the channel [05:49] rjack1 has joined the channel [05:49] rjack1: Hi All [05:50] rjack1: how to notify node.js when user leave the html page [05:51] rjack1: I have tried following code (JQuery): [05:51] rjack1: $(window).bind('unload', function(){ [05:51] rjack1:        $.ajax({cache: false, [05:51] rjack1:         dataType: "script", [05:51] rjack1:         async: false, [05:51] rjack1:         url: "http://node.example.com/unsubscribe" [05:51] rjack1:       }); [05:51] rjack1:     }); [05:51] rjack1: but it's not send request to node.js [05:52] okito has joined the channel [05:54] mikeal has joined the channel [05:56] cloudhead has joined the channel [06:04] rjack1 has left the channel [06:05] piranha has joined the channel [06:09] JimBastard_: lol rjack [06:22] aho has joined the channel [06:23] mikeal1 has joined the channel [06:23] joshholt_ has joined the channel [06:24] xer0x has joined the channel [06:37] mravaux has joined the channel [06:52] piranha has joined the channel [07:02] markwubben has joined the channel [07:07] kriskowal has joined the channel [07:14] dgathright has joined the channel [07:18] fizx has joined the channel [07:20] teemow has joined the channel [07:22] N` has joined the channel [07:24] mattly_ has joined the channel [07:27] brapse has joined the channel [07:33] felixge has joined the channel [07:35] romainhuet has joined the channel [07:38] felixge: _ry: the freelist patch looks like it may have a bad issue: http://github.com/ry/node/commit/57ea07ac912d88fb947a95ffd502c23f2d60f8b9 [07:38] derbumi has joined the channel [07:40] micheil: felixge: the code that's in freelist was just taken directly from net.js [07:40] felixge: micheil: it looks a little different [07:41] felixge: actually hm [07:41] micheil: it should be identical [07:41] felixge: I onlhy looked at http [07:41] felixge: which was different [07:41] micheil: I didn't do the changes to http, only net.js [07:41] tbassetto has joined the channel [07:41] felixge: anyway, I'm testing my suspicion now [07:41] micheil: _ry: added those in [07:41] felixge: micheil: ok, so if I'm right _ry is to blame :) [07:42] micheil: and yeah, I agree on the move from FreeList -> Freelist [07:42] micheil: I'm actually not sure if that code even works, I'm just assuming it must [07:43] micheil: I would've thought that Freelist.alloc would've checked the current length vs the max length.. but.. uh [07:43] micheil: or, rather [07:44] micheil: actually. no. it does work [07:45] felixge: freelist is broken [07:46] felixge: I have no idea how net works at all right now [07:46] felixge: let me show you [07:46] felixge: micheil: https://gist.github.com/26332fff906cced89179 [07:46] javajunky has joined the channel [07:46] felixge: this looks bad :) [07:47] micheil: hmm.. [07:47] felixge: _ry: you're not around anymore by any chance? [07:47] micheil: it's the scoping issue no? [07:48] micheil: felixge: the test is wrong [07:48] micheil: I think [07:48] micheil: the last one should be: !myClass.foo [07:48] felixge: micheil: no [07:49] felixge: micheil: the entire point of the test is to show that the constructor actually manipulates the 'freelist' [07:49] micheil: oh [07:49] felixge: rather than getting its own instance [07:50] felixge: changing line 13 in freelist to: this.constructor.apply({}, arguments) [07:50] felixge: could work [07:50] felixge: but I guess that still wouldn't make it inherit any of the prototype functions [07:50] felixge: this is *weird* [07:51] felixge: ah I see [07:51] felixge: http://github.com/ry/node/blob/57ea07ac912d88fb947a95ffd502c23f2d60f8b9/lib/net.js#L209 [07:51] felixge: ryan actually passes a factory method [07:51] felixge: not a constructor [07:52] felixge: *nvm* [07:52] felixge: I just misunderstood how this function is supposed to work [07:53] micheil: felixge: :) [07:53] felixge: :) [07:53] felixge: ACTION happy [07:53] felixge: :) [07:53] felixge: lets hope I can talk ryan into renaming this to 'factory' [07:53] micheil: felixge: like I said, the freelist stuff is taken directly from net.js, the original patch didn't change anything in http.js [07:54] felixge: micheil: you're right [07:54] felixge: micheil: I only scanned the top of the patch and was like "whuuuut" [07:54] felixge: :D [07:54] felixge: ok, gotta go. brb 15min [07:54] micheil: personally I think we should've staged the patch into two parts: spliting it from net.js, then adding it to http.js [07:55] towski has joined the channel [08:01] javajunky has joined the channel [08:09] TomY has joined the channel [08:10] nsm has joined the channel [08:11] felixge has joined the channel [08:11] felixge has joined the channel [08:16] mravaux has joined the channel [08:17] micheil: felixge: personally I think we should've staged the patch into two parts: spliting it from net.js, then adding it to http.js [08:19] felixge: micheil: right [08:20] dgathright has joined the channel [08:21] tisba has joined the channel [08:22] xla has joined the channel [08:22] hellp has joined the channel [08:23] dgathright has left the channel [08:28] tbassetto has joined the channel [08:29] kriskowal has joined the channel [08:33] kjeldahl has joined the channel [08:35] hellp has joined the channel [08:44] tbassetto1 has joined the channel [09:01] charlesjolley has joined the channel [09:17] gf3 has joined the channel [09:17] maritz has joined the channel [09:34] psynaptic has joined the channel [09:35] psynaptic has left the channel [09:37] toabi has joined the channel [09:37] toabi has left the channel [09:43] xla has joined the channel [09:48] botanicus has joined the channel [10:16] nsm has joined the channel [10:34] creationix has joined the channel [10:43] hassox has joined the channel [10:51] charlesjolley has joined the channel [10:56] mravaux has joined the channel [11:04] mape: Is there any module that allows you to fetch a page and use xpath/css selctors to match it? Know it is lacking a DOM but perhaps some magic is out there? [11:07] aho has joined the channel [11:09] N`: mape: maybe http://github.com/tautologistics/node-htmlparser [11:11] mape: Yeah tried that and it worked fine, only issue is finding the content is the mess of an object it outputs [11:11] mape: (really bad website that was parsed) [11:11] mape: tons of nested tables and all that fun stuff [11:13] mape: Guess I could go the regex path but doesn't feel optimal [11:20] joshbuddy has joined the channel [11:20] joshbuddy has joined the channel [11:56] alex-desktop has joined the channel [12:13] mravaux has joined the channel [12:14] MattJ has joined the channel [12:27] micheil has joined the channel [12:40] micheil: creationix: your websocket's server will like fail in the later versions of the spec, as the fields should be sent from the client in a random order [12:41] charlesjolley has joined the channel [12:47] jherdman has joined the channel [12:50] tek has joined the channel [12:52] confounds has joined the channel [13:08] TheEnd2012 has joined the channel [13:09] JAAulde has joined the channel [13:10] micheil: _ry: you about? [13:10] davidsklar has joined the channel [13:11] creationix: micheil: thanks, I'll update it [13:12] micheil: creationix: think it'd be possible to use the http_parser binding in a websockets implementation? [13:12] micheil: (with out requiring patches to node) [13:28] malkomalko has joined the channel [13:29] confounds has joined the channel [13:30] creationix: I don't think it's possible currently micheil but I know it's on the todo list for node. [13:31] micheil: where's the todo list? [13:31] creationix: In _ry's head I assume [13:31] micheil: oh, right [13:32] micheil: creationix: yeah, you'll also need to properly do the challenge auth [13:33] micheil: for if a browser / client doesn't receive the correct challenge token back, then it should reject the connection, iirc [13:35] javajunky: creationix: when's jsconf, I'm very time limited at the moment for updating articles on howtonode ? :( [13:36] micheil: jsconf is in.. [13:36] micheil: 4 days [13:36] javajunky: ;) ty. [13:55] mravaux_ has joined the channel [13:55] gf3 has joined the channel [14:09] creationix has joined the channel [14:10] mjr_ has joined the channel [14:15] dnolen has joined the channel [14:28] aryounce has joined the channel [14:30] alex-desktop has joined the channel [14:32] micheil: creationix: ping!~ping!@ :D [14:33] quirkey has joined the channel [14:34] micheil: creationix: given a little bit of work, and I think we'd be able to use Node's http_parser in the websocket implementations [14:35] creationix: micheil: cool [14:35] micheil: I just pulled in the http_parser binding into my websocket server [14:35] micheil: and then added one call back and was able to get the Request method [14:41] softdrink has joined the channel [14:41] joshbuddy has joined the channel [14:42] micheil: okay. this is epic. [14:44] nefD: ! [14:44] nefD: epic win? [14:45] micheil: yes [14:46] micheil: I've just taken the InComing request and Parsers freelist implementation from node's http.js, and gotten it to work by it's self [14:46] creationix: micheil: cool stuff, now what I really want is to be able to share a port with http requests and websocket requests [14:46] Yuffster has joined the channel [14:46] micheil: It could be possible I think. [14:46] micheil: that's how the spec was designed. [14:47] binary42 has joined the channel [14:47] binary42 has joined the channel [14:47] micheil: I think I'll make this http parser use the events system or something [14:49] mjr_: I've wired node's http parser up to raw packet capture data. [14:50] mjr_: Which is odd, but pretty handy. [14:50] micheil: yeah, I'm wanting to try and use it in websockets, which should be fun [14:53] fictorial has joined the channel [14:53] micheil: this sucks. I don't actually remember how to use the raw DOM to bind events./.. [14:55] sudoer has joined the channel [14:58] softdrink: element.addEventListener and element.attachEvent [14:58] micheil: cheers [14:58] softdrink: np [14:59] softdrink: (i just wrapped some of that code on a libraryless page hehe) [14:59] softdrink: first one takes 3 arguments [14:59] softdrink: attachEvent is for IE [15:00] micheil: oh man this is awesome. [15:00] micheil: I'm gunna have to move this to a branch just so I can share. [15:01] micheil: third event was bubble, no? [15:01] softdrink: yeah, i just pass false [15:02] softdrink: oh and IE expects "onclick" vs "click" [15:02] micheil: :| [15:02] micheil: just hit an error.. [15:02] softdrink: http://paste.mootools.net/m557a8510 [15:03] softdrink: there's my wrapper :) [15:03] sh1mmer has joined the channel [15:09] micheil: woot! got it working.. [15:09] micheil: this simplifies my websocket implementation heaps! [15:09] micheil: softdrink: yeah, I only need it to work in chrome, so I'm not fused. [15:10] softdrink: heh i envy you SO MUCH right now then [15:10] softdrink: if i only had to support webkit, i'd be the happiest web developer in the world [15:12] javajunky has joined the channel [15:12] micheil: softdrink: chrome !== webkit, persay [15:13] micheil: eg, webkit in chrome has rendering difference with webkit in safari [15:13] softdrink: yeah [15:13] softdrink: s/webkit/latest chrome dev builds [15:13] softdrink: :D [15:13] micheil: but no, the only reason this needs to support chrome is because it's using websockets and has a simple link on the page to trigger the sending of data to my server [15:14] micheil: and this hot chocolate is goooood... [15:14] micheil: :) [15:14] JAAulde has joined the channel [15:17] aryounce has joined the channel [15:22] hsuh has joined the channel [15:24] brapse has joined the channel [15:25] creationix: woot, fast, lean, git based fs [15:25] creationix: implementing smart and correct caches is tricky [15:26] micheil: creationix: using the new Freelist implementation? [15:26] micheil: / used? [15:26] creationix: no, what's that [15:26] creationix: I just did this http://github.com/creationix/node-git/blob/master/lib/git.js [15:27] micheil: it's in the latest HEAD, lib/freelist.js [15:27] deanlandolt: voodootikigod: can i send you an updated description of my talk? [15:27] voodootikigod: for track b [15:27] voodootikigod: yes [15:27] deanlandolt: sweet...thanks [15:28] micheil: voodootikigod: hmm.. jsconf? [15:28] voodootikigod: ? [15:28] nefD has joined the channel [15:28] micheil: voodootikigod: track b as in jsconf track b? [15:29] voodootikigod: y [15:29] micheil: ah, cool [15:29] creationix: micheil: what's the point of freelist? [15:29] micheil: It would've been good to go, but alas, exams got in the way. (thank god this is my last year of school.) [15:29] micheil: creationix: to save you from creating too many processes [15:30] creationix: so it's some sort of throttle? [15:31] micheil: sorta [15:32] micheil: check out how net.js uses it [15:32] micheil: it's a Factory [15:32] micheil: new Freelist(name, max_size, factory) [15:33] micheil: then when you need to use an instance of factory, Freelist.alloc(args for factory) [15:33] micheil: (Freelist in that instance being a stored Freelist instance) [15:33] confounds has left the channel [15:33] micheil: then when you're done with it, Freelist.free() [15:33] micheil: erm [15:34] micheil: Freelist_instance.free(current_alloc) [15:34] micheil: I think we can take these further by adding an extra arg to the constructor that is what to do onFree [15:35] creationix: It doesn't look like the free method is freeing anything [15:35] micheil: ie, var myFreeList = new Freelist("parsers", 100, function(){...}, function(){...} [15:35] creationix: it's adding to the list if there is room [15:35] micheil: well, it must work [15:35] creationix: well, I need a different kind of cache and semaphore [15:35] micheil: it's not really free'ing anything [15:36] creationix: I don't ever want two subprocesses to run at the same time with the same arguments [15:36] micheil: it's really actually just limiting number of factory instances created [15:36] micheil: _ry: Proposal, rename FreeList => Factory [15:37] micheil: so, it'd be var parsers = new Factory(max, factory_initializer, factory_onFree); [15:39] justinlilly has left the channel [15:42] creationix: micheil: I see now [15:42] creationix: it's not a throttle at all, it's an object pool [15:42] creationix: "pool" like in thread pool [15:43] micheil: yeah [15:43] creationix: alloc either pulls one out of the pool or creates a new one [15:43] creationix: and free puts one back in the pool if there is room [15:43] micheil: yep [15:43] creationix: but it doesn't limit the number that exist in any way, just the number of idle objects in the pool [15:44] micheil: so what would you call it? [15:44] creationix: I think FreeList is a good name, though the JS could use some optimizing [15:44] creationix: and maybe a line or two of comments explaining what it's intended for [15:45] micheil: well, I'm adding in stuff for a patch to have it have an onFree callback [15:45] creationix: an onFree option doesn't make any sense [15:45] creationix: it's outside the scope of this utility [15:45] micheil: not really [15:45] creationix: free completely ignores your object if the pool is full [15:46] micheil: because were it's used it's needed [15:46] micheil: rather then having to tell the parser to reinitialize after it's just been created [15:47] micheil: actually. no, I see what you mean. [15:48] dandean has joined the channel [15:51] creationix: Is it fair to say that alloc and free will be called many more times that new FreeList [15:51] creationix: if so, I think this would be faster in functional style [15:52] steadicat has joined the channel [15:54] micheil: hmm? [15:54] RayMorgan has joined the channel [15:54] creationix: micheil: https://gist.github.com/55c1f0593dd8f5c6c49f [15:54] creationix: easier to read too I think [15:54] micheil: yeah, good point [15:55] micheil: why list[list.length] = ? [15:55] creationix: is cost more to create the alloc and free functions every time you make a new FreeList, but once you make one, there are less lookups to this.foo [15:55] creationix: that's faster than push [15:55] micheil: oh [15:56] micheil: does it work the same? [15:56] creationix: yep [15:56] creationix: arrays are magical [15:56] micheil: what about the list.pop [15:56] jbrantly: creationix: do you know offhand if thats faster than push across browsers too? [15:56] micheil: it was list.shift() [15:56] creationix: and shift is horribly slow, so I changed it to pop [15:56] micheil: okay [15:56] jbrantly: creationix: never thought to benchmark that [15:56] JAAulde has joined the channel [15:56] creationix: jbrantly: I think it is, but I haven't tested in a while [15:57] creationix: the objects in the pool won't get used uniformy, but that may be a good thing [15:57] jbrantly: creationix: something interesting to look into. Thanks. [15:57] creationix: the ones in the end will be hotter and more likely to be in more caches [15:57] micheil: testing.. [15:57] micheil: works. [15:58] creationix: I'm not 100% sure that pulling name, max, list, and constructor from the closure is faster than this. loockups [15:58] creationix: but I am sure it's easier to read [15:59] micheil: it looks better. [15:59] micheil: do you want this patch, or can I take it? [15:59] sveisvei has joined the channel [15:59] micheil: (I've got another patch on top to remove name, as it's never used) [16:00] creationix: you can have it [16:00] micheil: okay, thanks [16:00] creationix: _ry probably has something in mind for name, just ask him [16:01] creationix: note that I changed the api a little [16:01] creationix: var FreeList = require('freelist') not require('freelist').FreeList [16:02] creationix: and to create one it's just "FreeList(...)" not "new FreeList(...)" [16:02] micheil: hmm.. [16:02] micheil: new FreeList worked [16:02] creationix: it works, but it's usless overhead [16:02] micheil: yeah, but you want to maintain instances [16:03] creationix: it's a new instance of object every time [16:03] creationix: that's what {} literals do [16:03] creationix: except for inheritance stuff, "new" is syntatic suger" [16:03] micheil: oh? [16:03] bpot has joined the channel [16:04] micheil: so two Freelist()'s shouldn't share the same list[] yeah? [16:04] creationix: correct, it's safe [16:04] creationix: it's a local private variable [16:04] micheil: k [16:04] creationix: and since you're not using "new" to call the function, it's very bad to use "this.foo = bar" inside it [16:04] creationix: then this will most likely be "global" [16:05] micheil: hmm.. I don't think that should have any negative impacts, I'll test. [16:06] creationix: hmm, I see where name was used [16:06] micheil: ah, yes, so do I know [16:06] micheil: *now [16:06] micheil: in the debugs [16:06] creationix: ok, updated the gist [16:07] creationix: oh, you're right, I didn't update the code in the debugs [16:07] creationix: updated again [16:09] lifo has joined the channel [16:10] micheil: works fine., [16:10] micheil: creating patch. [16:11] creationix: you'll have to update all the code that uses it since I changed the api [16:11] creationix: or he won't take the patch [16:11] micheil: done that. [16:11] micheil: _ry: https://gist.github.com/550269276be5e840eb52 [16:12] creationix: micheil: have you ever used node-bench [16:12] micheil: nup [16:13] creationix: I would highly recomment benchmarking the two versions side by side and link to the results [16:13] creationix: see this example from my last patch https://gist.github.com/bdd159b4f8f7c900d939 [16:13] creationix: http://github.com/isaacs/node-bench [16:14] creationix: I'm not even sure the new version is faster, it should be, but I'd like to be sure [16:14] micheil: umm.. how would I be able to benchmark the freelist stuff.. [16:14] creationix: paste the two implementations in a file with different names [16:15] creationix: then create a typical use case that creates 10 or so factories and hundreds of instances [16:15] creationix: it would be tricky for sure [16:16] creationix: maybe just benchmark the changes by themselves [16:16] creationix: do one for array push vs the length trick [16:16] creationix: one for shift vs pop [16:16] micheil: no thanks. As much as benchmarking would be great, that seems too complicated [16:16] micheil: creationix: remember: Correct, Beautiful, Fast [16:17] creationix: yes, it's correct, it's beautiful, now to see if it's fast [16:17] creationix: this is going into core node where speed really matters [16:18] micheil: i think it's faster [16:18] creationix: alright then, I'm going back to work though :) [16:20] micheil: same. [16:20] micheil: I'm wanting to get my HTTP Parser branch on node-websocket-server merged by tonight [16:21] micheil: creationix: http://github.com/miksago/node-websocket-server/blob/http_parser/lib/ws/connection.js#L15-37 [16:22] creationix: cool [16:22] micheil: admittedly I've just ripped the parser out from http.js are modded it, but nonetheless [16:24] creationix: alright 3402.81 reqs/sec, now we're in business again [16:24] creationix: sys.inspect is slow, I had one in there and it dropped my performance down to about 200/sec [16:25] isaacs has joined the channel [16:26] indiefan has joined the channel [16:27] mjr_: It depends on what you are sys.inspecting, I'd imagine. [16:27] micheil: blooody hell that's a lot. [16:28] JAAulde has joined the channel [16:29] nefD: hrmm relying on dbslayer for async mysql access in the long term is scary [16:30] _ry: 'lo [16:31] nefD: allo, ry [16:32] isaacs: "[16:18] micheil: i think it's faster" [16:32] isaacs: famous last words [16:32] micheil: lol [16:33] technoweenie has joined the channel [16:33] micheil: either way, it doesn't really matter that much as to whether the patch is used or not [16:33] creationix: no, but that shift does need to be replaced with a pop [16:33] creationix: I KNOW that's faster [16:34] creationix: the rest of it is iffy [16:34] creationix: hmm, I wonder if buffers are faster than strings [16:35] mattly has joined the channel [16:35] creationix: I'm caching the result in ram as a string, but it's still really slow for large message bodies [16:35] inimino: ACTION would love to see some buffer-vs-string benchmarks [16:36] creationix: I think it's faster ;) [16:36] creationix: ACTION looks up Buffer api [16:36] creationix: I'm loving having up-to-date docs [16:38] creationix: so what happens when you copy a unicode string into a buffer? [16:38] creationix: ACTION opens up node-repl [16:38] micheil: creationix: buffers are pretty fast, I'm using them in that websocket implementation [16:38] nefD: hmm this libmysql async mysql module is coming along pretty well, actually.. maybe I don't need to rely on dbslayer after all [16:38] _ry: micheil: i like my freelist better [16:39] micheil: _ry: okay :) [16:39] creationix: _ry: shift is really slow, it updates every index in the array by one [16:40] creationix: also if you use pop, then the object on the end of the list will ne "hotter" in memory and more likely to cache [16:40] creationix: but yeah, your style is fine, it matches the rest of your apis [16:41] isaacs: note that push, pop, shift, and unshift are all slower than keeping track of the array length yourself, and assigning to that while you increment or decrement it [16:42] creationix: hmm, how do I copy an utf8 string into a new buffer, length gives the length in characters, not bytes [16:42] charlesjolley has joined the channel [16:42] creationix: I don't know how large to make the buffer [16:42] isaacs: creationix: Buffer.utf8Length? [16:43] creationix: where is that, I don't see it? [16:43] isaacs: oh, no Buffer.byteLength(str) [16:43] isaacs: it changed names from when i played with it in net2. or my brain changed the name in my memory [16:44] creationix: ahh, that needs to be documented [16:44] creationix: it's missing from the docs [16:44] creationix: sweet, works [16:44] creationix: now to benchmark it! [16:45] isaacs: function stob (str) { var b = new Buffer(Buffer.byteLength(str)); b.utf8Write(str); return b } [16:45] creationix: that's handy [16:45] isaacs: hm. that should be in the buffer module, imo. [16:45] creationix: agreed [16:45] isaacs: creationix: go do that! [16:46] creationix: lol, not today, I've got to get this done this week [16:46] isaacs: or maybe a Buffer.fromString(str) that returns a new buffer. [16:46] creationix: maybe a general buffer library with conversions to and from most data types (utf strings, unsigned integers, etc...) [16:52] voodootikigod_ has joined the channel [16:52] javajunky: ughhh stupid issues.apache.org being compromised :( [16:57] joshthecoder has joined the channel [16:57] joshthecoder: hello, any idea why this error is happening? https://gist.github.com/bdd7ff902eb5d27fe175 [16:59] creationix: isaacs: yeah, buffers are faster. my 178k response pulled from an in-memory cache renders 296/sec if I store the cache as a string, and 1516/sec if I store the cache as a buffer [16:59] creationix: ACTION is happy [16:59] micheil: good damn is that a lot faster [17:00] isaacs: ACTION <3 benchmarks! [17:00] isaacs: javajunky: inorite?! when i first signed up and they emailed me my password, i was like, "aw, damn. well, this won't be long" [17:01] _ry: joshthecoder: um.. [17:01] javajunky: javajunky: yeah ironically and painfully they just e-mailed me the new one *sobbity* [17:01] _ry: joshthecoder: rm -rf build [17:01] _ry: joshthecoder: then reconfigure [17:01] javajunky: isaacs: er yeah that was meant for you, I should sleep :) [17:02] joshthecoder: _ry, works now, thanks [17:02] joshthecoder: opps spoke too soon, errors out again [17:03] creationix: _ry: thanks for buffers, they saved my performance bottleneck [17:04] _ry: raw memory ftw :) [17:04] BinaryPie has joined the channel [17:05] creationix: I can handle 1600/sec for a page that big [17:05] creationix: bandwidth will clearly be the bottleneck on a real site there [17:09] charlesjolley has joined the channel [17:11] avidal: heh [17:11] avidal: process.env.NODE_ALLOW_STUPID [17:12] isaacs: avidal: :) [17:14] tisba_ has joined the channel [17:14] isaacs: avidal: i think it'd be ideal if, when you do NODE_ALLOW_STUPID=2 node program.js; it immediatly spits out to stdout "What are you, stupid or something!?" [17:14] avidal: heh [17:14] avidal: yeah, that'd be interesting [17:14] isaacs: not stderr [17:14] isaacs: that'd be too easy to ignore [17:15] avidal: prepends 'I AM STUPID HURRR' to every write, socket or otherwise [17:15] isaacs: hahahah [17:15] mjr_ has joined the channel [17:18] dnolen has joined the channel [17:21] jpoz has joined the channel [17:23] avidal: hrm, it'd be interesting if chat.nodejs.org ran as an irc proxy of sorts [17:23] avidal: like mibbit [17:23] avidal: but it only went in here [17:24] nsm has joined the channel [17:25] avidal: for that, i suppose the server would need to maintain a freenode connection for each connected client [17:25] isaacs: avidal: or it could just maintain one freenode connection for a bot [17:25] avidal: yeah, i suppose [17:25] isaacs: and that bot could just prefix the messages with the username [17:25] avidal: but that's not nearly as cool [17:26] isaacs: avidal: it's still pretty cool, i think [17:26] nefD: woot.. child_process for the *win* [17:35] mattly has joined the channel [17:36] charlesjolley has joined the channel [17:39] ssteinerX has joined the channel [17:48] javajunky has joined the channel [17:48] Iveaux has joined the channel [17:48] cadorn has joined the channel [17:50] devinus has joined the channel [17:59] charlesjolley has joined the channel [18:02] batasrki has joined the channel [18:03] stephenlb has joined the channel [18:03] mikeal has joined the channel [18:09] javajunky has joined the channel [18:17] drostie has joined the channel [18:17] micheil: _ry: I've got an issue with the http parser.. [18:17] tilgovi has joined the channel [18:19] micheil: _ry: I'm never receiving the onData call back [18:19] micheil: rather, onBody [18:21] bmizerany has joined the channel [18:24] brianmario has joined the channel [18:26] admc has joined the channel [18:27] softdrink has joined the channel [18:36] jpoz has joined the channel [18:38] brapse has joined the channel [18:40] towski has joined the channel [18:42] fictorial: I'm confusing myself re: process.nextTick. When would one want to use process.nextTick(someFunction) instead of just invoking someFunction immediately? What would run between the start of the next tick and someFunction if I used nextTick? A check for r/w events? [18:43] mjr_: The thing that woke you up might also be about to wake somebody else up on that same trip through the event loop [18:44] fictorial: Not sure that I have anything waking me up though. Hmm [18:44] fictorial: in that context I mean. [18:44] fictorial: My specific use case is that I have many clients, each with a queue of commands that are handled asynchronously. When any one command returns I want to carry on with the next __for that client__. I am debating handling the next in the queue immediately (but async still) or in a nextTick but, well, I'm not sure why. :) [18:49] _ry: fictorial: for faking async [18:49] _ry: fictorial: to provide the same interface for things that are either async or not async [18:49] _ry: e.g. (look up this domain name - which might be an ip address) [18:49] micheil: _ry: should http_parser return the body at all? [18:50] _ry: micheil: what do you mean? [18:50] _ry: yes? [18:50] micheil: well, I have a request, which is a GET request, I'm using the code from the http.js's parser to parse the data on the socket [18:51] micheil: although, the onBody event on http_parser never gets fired [18:51] _ry: maybe the get isn't well formed [18:51] konobi: there isn't a body on GET [18:51] micheil: it should be [18:51] micheil: konobi: there is in websockets [18:51] _ry: (e.g. doesn't have a content length?) [18:51] konobi: werid [18:51] micheil: so, you'd need to send a content-length for it to work?> [18:52] _ry: yes - http assumes 0 body without content-length [18:52] _ry: (it's more complicated than that, but approximately) [18:52] micheil: hm.. [18:52] micheil: okay [18:53] micheil: is that by the standard? [18:53] _ry: micheil: yes [18:53] micheil: okay. I'll make note of that. [18:53] _ry: it also depends on what version of http [18:53] _ry: 1.1 defaults to keep alive [18:53] _ry: 1.0 doesn't [18:53] micheil: 1.1 it is [18:53] _ry: so with 1.0 you can send a body without a content-length [18:53] jpoz has joined the channel [18:53] micheil: don't you mean 1.1? [18:54] inimino: no, 1.0 [18:54] _ry: no [18:54] micheil: okay [18:54] inimino: but then you close the connection after [18:54] inimino: so content-length is optional [18:54] _ry: right the body is terminate by the TCP EOF [18:54] jbrantly has left the channel [18:54] mikeal has joined the channel [18:54] _ry: you can do that in 1.1 too if you use "Connection: close [18:54] _ry: http is fucked. [18:54] mikeal: wow, that's the second time i crashed this entire Mac tracking down this bug in node [18:55] _ry: mikeal: yeah i've crashed my mac too [18:55] mikeal: it's totally random [18:55] _ry: what a fucking peice of shit operating system [18:55] mikeal: i ran the same code again and it's fine [18:55] mikeal: under the hood it's a lot of shoestring and duct tape [18:55] _ry: totally [18:55] mikeal: the filesystem is the worst tho [18:55] javajunky has joined the channel [18:55] _ry: it's worse than freebsd [18:55] mikeal: just fucking dreadful [18:56] _ry: i really can't believe people buy apple products [18:56] isaacs: i don't know why some company isn't making high-end all-metal laptops for ubuntu [18:56] mikeal: i can't reproduce this bug in isolation [18:56] mikeal: the ordering is so specific [18:56] isaacs: there's clearly a market [18:56] deanlandolt: _ry: they're /really/ shiny, that's why ;) [18:56] mikeal: i literally reproduce all the requests in pure node and I can't expose it [18:56] kriskowal has joined the channel [18:56] mikeal: _ry: can you just come over to couch.io office :) [18:56] fictorial: I buy apple products to develop apps for others with twinkles in their eyes, that's about it :) [18:56] fictorial: _ry: thanks, I see the lib/dns.js nextTick usage but I don't follow you. The interface is still the same regardless of whether nextTick is used or not. [18:56] mikeal: i'll buy you lunch, and dinner, and drinks :) [18:57] _ry: mikeal: are you coming to jsconf? [18:57] mikeal: nope, missed the ticket deadline [18:57] mikeal: i really wish I was going [18:57] micheil: mikeal: nothing would stop an impromptu node.js meetup [18:57] isaacs: fictorial: it's not quite the same. the cb will be called after the invocation if you use process.nextTick, or right AT the invocation if you do'nt. [18:57] micheil: / _ry ^^ [18:58] deanlandolt: i heard something about a room dedicated to node.js hacking on friday [18:58] mikeal: i wonder if this bug is actually even happening in linux [18:58] mikeal: hrm...... [18:58] mikeal: if it's not I can stop caring about it [18:58] micheil: _ry: would it be possible for me to fake the content-length? [18:58] fictorial: isaacs: right, I follow that but the interface is the same to the caller. Why not just callback immediately? I am missing something. [18:58] _ry: mikeal: why [18:58] _ry: er [18:58] _ry: wrong user [18:58] _ry: micheil: why [18:59] isaacs: fictorial: doSomething(cb); butDoThisFirst(); [18:59] _ry: micheil: you can send transfer-encoding: chunked [18:59] _ry: micheil: which doesn't require a content-length [18:59] fictorial: isaacs: ... aha! ok [18:59] isaacs: fictorial: if doSomething() does cb(), then the cb will run before butDoThisFirst() [18:59] micheil: _ry: because, the websocket protocol doesn't send either content-length or transfer-encoding [18:59] _ry: micheil: well it's also not http, right? :) [18:59] micheil: ie, could I add a header in somehow? [18:59] isaacs: fictorial: nextTick puts the cb at the END of the execution thread [18:59] fictorial: isaacs: I wasn't thinking of it that way... I tend to put butDoThisFirst() stuff ahead of async calls. Heh. [19:00] pjb3 has joined the channel [19:00] isaacs: fictorial: sure. it's an edge case, but still. [19:00] fictorial: isaacs: Yep, now I am with you. Thanks. [19:00] pjb3 has left the channel [19:00] mjr_: I noticed that nextTick runs after setTimeout(fn, 1), which surprised me. [19:01] _ry: mjr_: huh? [19:01] _ry: hm [19:01] mikeal: ls [19:01] mikeal: whoops [19:01] mjr_: lemme do it again [19:01] mikeal: wrong window [19:01] jbrantly has joined the channel [19:01] _ry: mjr_: it's using a high priority idle watcher.. which supposely should come first [19:02] isaacs: mjr_: i'm pretty sure it's not supposed to work like that. nextTick should run before setTimeout(, 0) [19:02] _ry: we should add a test [19:02] towski has joined the channel [19:03] mikeal: most js vm's have bugs in setTimeout(,0) [19:03] mikeal: someone wrote it up recently, i think it was resig or ajaxian or something [19:03] _ry: well, luckly we control that completely so we can determine it's behavior [19:03] _ry: s/'// [19:03] mikeal: awesome [19:03] isaacs: mikeal: it was resig. windows has the round-to-50-ms issue [19:03] isaacs: mikeal: ajaxian doesn't "write" anything. they just copy. [19:03] isaacs: mikeal: sorry, i mean, "syndicate" [19:04] mjr_: http://gist.github.com/364949 [19:04] mikeal: hahaha [19:04] _ry: mjr_: yeah that's not good. [19:04] mjr_: Does this not look like setTimeout runs before nextTick? [19:05] isaacs: mjr_: hehe [19:05] isaacs: mjr_: try replacing "puts" with "debug" [19:05] isaacs: puts is not synchronous [19:05] creationix: _ry: how do I use buffer.copy? the docs don't say what the four arguments mean [19:05] _ry: isaacs: they're not sync but they're ordered [19:05] isaacs: _ry: ok, then it's broken. [19:06] mjr_: Check it again: http://gist.github.com/364949 [19:06] _ry: creationix: um.. hmm [19:06] isaacs: ok, any way you slice it, that's a bug [19:06] deanlandolt: isaacs: don't be bitter -- you know you were thrilled to be "syndicated" ;) [19:06] _ry: creationix: seems clear to me :) [19:06] mjr_: Not sure what the threshold is in there, but eventually the setTimeout does seem to set a timer. [19:06] isaacs: deanlandolt: true, but less thrilled to not be credited. [19:07] creationix: _ry, all, I think I figured it out, I don't need the last two arguments if I want to copy the whole thing [19:07] deanlandolt: yeah i know...that's bullshit [19:07] _ry: creationix: yeah [19:07] _ry: creationix: btw, feel free to patch the docs [19:07] _ry: if you have a cute little example or something [19:08] creationix: _ry: I'm taking notes on the parts that aren't clear so I can patch it when I'm done [19:08] mikeal: dammit, i get it in linux too [19:08] _ry: creationix: cool [19:08] _ry: mikeal: that's good [19:09] _ry: reproducable = godo [19:09] mikeal: i'll show you the code real quick [19:09] jpoz has joined the channel [19:09] creationix: and I'm getting consistent 2000+ reqs/sec on real application pages even with very short lived caches (100ms) [19:09] mikeal: _ry: http://gist.github.com/364961 [19:10] mikeal: it's a dirt simple proxy server that removes all the headers from client and server [19:10] mikeal: so that I can try to track down what is actually causing the problem [19:10] mikeal: and all I do is trigger a couchdb replication through it [19:11] mikeal: oh, ignore doRequest [19:11] fizx has joined the channel [19:11] mikeal: that was for some other tests i was doing [19:12] micheil: I'm sure hoping I'm not going to need to revert to the old header parser I was using in this websockets implementation [19:13] mikeal: added the exception to the comments on the gist [19:14] mjr_: _ry: I created issue 99 on github for the timing thing, since it seems like that's an issue for you. [19:14] mikeal: making the headers empty still shows the bug [19:15] mikeal: when i try to reproduce the problem by doing the requests myself I can't expose the bug [19:15] mikeal: it's only when I actually run couchdb replication through it, which is SUPER annoying [19:17] tilgovi has joined the channel [19:21] CodeOfficer has joined the channel [19:22] isaacs: hey, i'm seeing failures on test/simple/test-http-head-request.js [19:23] creationix: isaacs: it's a know bug, see the commit for that file [19:23] creationix: *known [19:23] isaacs: creationix: awesome, thanks [19:24] maushu has joined the channel [19:25] isaacs: oh... i see. the setTimeout is keeping it from exiting, since there's something on the event loop, i take it? [19:27] sh1mmer has joined the channel [19:27] nefD: hmm.. if I send a SIGINT signal to a running node process, with that script wait until all currently running events are completed before exiting? [19:28] MattJ has joined the channel [19:30] _ry: mjr_: thanks [19:31] creationix: nefD, I know it will stop idle waiters like setTimeouts, not sure about things already queued for execution though [19:31] _ry: isaacs: yeah head requests are broken [19:32] nefD: creationix: Ah, gotcha, thanks.. im mostly curious about any pending http requests which have been received and are being processed.. specifically, whether those will be completed and returned prior to exiting [19:32] creationix: I imagine if they have been received, but are waiting on some async call inside them, then it won't finish [19:33] nefD: *nod* I was thinking the same, just didn't have a handy way to test it in a quick manner :P [19:33] nefD: though, it appears I can attach listeners to signals, so I can likely handle it manually if need be [19:33] creationix: yeah [19:34] siong1987 has joined the channel [19:37] stepheneb has joined the channel [19:37] okito has joined the channel [19:39] nefD: child.kill is a very misleading method name :P [19:41] nefD: child.signal makes more sense, imho [19:42] mjr_: Tell that to the fine folks who made the unix syscall kill() [19:42] mjr_: But yes, I agree. [19:42] isaacs: i think it's because unix is inherently autistic and antisocial. [19:42] KungFuHamster: send the process home to the sea, like they do in Sinanju [19:42] isaacs: communication feels like an attack. [19:43] felixge has joined the channel [19:43] steadicat has joined the channel [19:44] nefD: Unix: "My girlfriend and I have been having problems, so I asked her if we could SIGHUP to save the relationship. Thats when she gave me the SIGTERM." [19:45] isaacs: nefD: you're lucky she didn't try to -9 you [19:45] nefD: indeed.. [19:45] mjr_: Or close your standard output, if you know what I mean. [19:45] isaacs: hahaha [19:45] nefD: lol [19:46] mjr_: note that I do not know what that means. [19:46] isaacs: mjr_: i have several ideas, and they're all pretty funny. so you just made like 4 jokes. it's polymorphic comedy. [19:47] mjr_: I like to think that my joke can be used as a prototype for other jokes, rather than setting up a tedious joke hierarchy. [19:47] nefD: XD [19:48] mattly has joined the channel [19:49] KungFuHamster: as long as you don't turn a private function into a public one [19:50] nefD: The moral of the story is to make sure your attributes are always protected [19:52] codehero: hi all, any pointers on doing one shot event listeners? [19:53] isaacs: holy crap we're nerd.s [19:53] stephenlb: heh [19:53] nefD: ACTION pushes his glasses up [19:54] codehero: I am a geek [19:56] towski has joined the channel [20:00] mikeal: _ry: is there any reason that the Client constructor would be called on a socket that is being used in a ServerResponse? [20:14] xla has joined the channel [20:15] dgathright has joined the channel [20:15] hellp has joined the channel [20:18] charlesjolley has joined the channel [20:20] siong1987 has joined the channel [20:24] JimBastard has joined the channel [20:25] RayMorgan_ has joined the channel [20:25] JimBastard: hey RayMorgan you around? was wondering if it was possible to use Mu synchronously if i wanted to [20:25] JimBastard: like is that even an option in Mu? [20:27] xla has joined the channel [20:30] JimBastard: weak [20:31] _ry: mikeal: no [20:31] _ry: mikeal: well, at least i don't think so [20:32] mikeal: ok, i'm slowly starting to figure what the hell this code does :) [20:33] mikeal: _ry: these request are pipelined [20:35] _ry: ok [20:35] siong1987 has joined the channel [20:35] mikeal: that might have something to do with it [20:35] _ry: the requests to couch or the outside requests? [20:36] _ry: i guess the outside :) [20:36] _ry: since node doesn't do keep alive on client [20:37] mikeal: i have a print inside ondata for Client that checks for the fd that I'm later throwing on [20:39] sh1mmer has joined the channel [20:42] nefD: I like the new docs [20:42] mikeal: ok, now I'm starting to understand this a bit better [20:45] CIA-77: node: 03Ryan Dahl 07master * r6bdb4f7 10/ (src/node.js test/simple/test-process-mixin.js): process.mixin: deprication -> removed - http://bit.ly/bTSveq [20:45] _ry: i spelt it wrong again. [20:46] CIA-77: node: 03Ryan Dahl 07master * rb98cd67 10/ (src/node.js test/simple/test-process-mixin.js): process.mixin: deprecation -> removed - http://bit.ly/bxjgp0 [20:47] nefD: heh [20:47] nefD: bugfix: commit typo [20:48] mjijackson has joined the channel [20:49] _ry: 2 files changed, 1 insertions(+), 113 deletions(-) [20:49] _ry: feels so great [20:49] mjr_: _ry: I noticed that you didn't pull my doc fix that removed the reference to process.mixin in api.markdown [20:49] mjr_: maybe that bit should be removed from the docs now. [20:50] sh1mmer has joined the channel [20:50] CIA-77: node: 03Ryan Dahl 07master * r57edff1 10/ doc/api.markdown : Remove mixin from docs - http://bit.ly/9uwOX7 [20:50] _ry: mjr_: oh thanks [20:50] mjr_: boom [20:51] xla has left the channel [20:54] isaacs: _ry: w00t. [20:54] isaacs: nice [20:56] _ry: rhys just sent an update that he's trying to have ossl stuff ready for jsconf [20:58] felixge has joined the channel [20:58] felixge has joined the channel [21:05] xla has joined the channel [21:06] xla has left the channel [21:08] dnolen has joined the channel [21:08] gwoo has joined the channel [21:12] mattly has joined the channel [21:12] darkf has joined the channel [21:13] xla has joined the channel [21:13] xla has left the channel [21:15] xla has joined the channel [21:16] xla has left the channel [21:24] admc has joined the channel [21:25] softdrink has joined the channel [21:31] RayMorgan has joined the channel [21:34] avidal: hrrrm [21:39] fictorial: nice, flow-js just made my life easier [21:47] joshbuddy has joined the channel [21:55] RayMorgan has joined the channel [21:56] bmizerany has joined the channel [21:57] mikeal: isaacs: what? [21:58] xla has joined the channel [21:58] isaacs: mikeal: what what? [21:58] mikeal: the twitter post [21:58] mikeal: i'm confused [21:58] derbumi has joined the channel [21:59] mattly: is there a reason http.ClientResponse's headers are all lowercase instead of the standard uppercase? [21:59] isaacs: mikeal: re: http://twitter.com/mikeal/status/12117539676 [21:59] mikeal: i see [21:59] mikeal: now i get it [21:59] xla has joined the channel [21:59] mikeal: _ry: i *finally* tracked down this bug [21:59] mikeal: so, i'm getting a data event [22:00] isaacs: mikeal: because i LOVE it when people reply to thier own problems with that [22:00] mikeal: with a chunk that does not equal 42 until it hits [22:00] mikeal: http://github.com/ry/node/blob/master/lib/http.js#L336 [22:02] sveisvei has joined the channel [22:07] mikeal: _ry: what is that line suppose to do [22:07] mikeal: ? [22:08] maushu: _ry, who do I have to bribe/kill to have access to contexts in node.js? [22:09] isaacs: maushu: the es5 committee [22:09] steadicat has joined the channel [22:09] isaacs: maushu: or, if you'd like, you could push for the module refactor, and replace Module.prototype.postCompile to use evalcx instead of process.compile [22:09] maushu: I mean v8 contexts in node.js. [22:09] isaacs: maushu: you have evalcx, right? [22:09] isaacs: what are you trying to do? [22:09] _ry: mikeal: that sends the chunk size [22:10] maushu: What we do every night, pinky. [22:10] isaacs: heh [22:10] maushu: Create sandbox! [22:10] mikeal: ok, why the (16) ? [22:10] _ry: mikeal: hex [22:10] isaacs: maushu: you can do that with process.evalcx [22:10] mikeal: the .length is 66, but .length.toString(16) is 42 [22:10] isaacs: maushu: is that not sufficient for your purposes? [22:10] mikeal: which is END_OF_FILE [22:10] _ry: mikeal: hmm [22:10] maushu: What about infinite loops? [22:11] isaacs: maushu: then you need a child process. [22:11] isaacs: maushu: spawn a child process, pipe the code into it, it runs it in a sandbox, if it doesn't return in 30 seconds, you kill it [22:11] _ry: mikeal: are you saying that my arbitrary choice of END_OF_FILE is fucking shit up? [22:11] maushu: Doesn't v8 context handle infinite loops? [22:11] mikeal: in this case, yes :) [22:11] _ry: mikeal: what if you set END_OF_FILE to 999999 [22:11] isaacs: maushu: no, that's on the application to do. chrome does this by putting each tab in a different process. [22:12] isaacs: er, thread? [22:12] mikeal: lemme see [22:12] maushu: Ah, drat. [22:12] _ry: isaacs: process [22:12] maushu: So contexts are useless in my case. [22:12] isaacs: i dunno, most of my chrome knowledge comes from that comic book they sent out [22:12] isaacs: maushu: not necessarily [22:12] _ry: love processes [22:12] _ry: they will love you [22:12] isaacs: maushu: protecting against infinite looping can be done with child procs [22:12] maushu: Yes, because using child processes is what Im doing right now. [22:12] avidal: hrm, trying to trade my n900 [22:13] isaacs: maushu: but you still need to prevent require("fs") monkeying [22:13] maushu: Done. [22:13] maushu: The problem is that each process is 2mb. I believe. [22:14] maushu: Having a process for each sandbox makes me twitch. [22:14] mikeal: _ry: yup! [22:14] mikeal: oh my god [22:14] _ry: mikeal: crazy [22:14] mikeal: it only took 2 days and a lunch time beer to figure this out [22:14] _ry: mikeal: but now we need to find out how that's getting compared [22:15] mikeal: what do you mean? [22:15] maushu: _ry, so v8 contexts don't handle infinite loops and alike? [22:15] _ry: god.. [22:15] mikeal: so, that one line fix just made all my problems go away [22:15] _ry: is this http://github.com/ry/node/blob/master/lib/net.js#L385 coercing a buffer into an integer? [22:16] mikeal: i'm doing a full replication [22:16] mikeal: yup :) [22:16] _ry: that's horrible [22:16] _ry: maushu: 'handle' ? [22:16] mikeal: you could just string the EOF [22:16] mikeal: and then do a === compare [22:16] mikeal: oh wait, that won't work [22:16] maushu: _ry, basically, if a context freezes do the other contexts freeze too? [22:17] derbumi has joined the channel [22:17] _ry: mikeal: can you see if you change it to 42 again [22:17] mikeal: because it's usually END_OF_FILE + '\n\r' [22:17] isaacs: maushu: when you do script.compile(), it runs until it's done. [22:17] _ry: and change it to === ? [22:17] mikeal: that won't ever find it tho [22:17] isaacs: maushu: since v8 doesn't know that the loop is infinite, or that you didn't mean for it to hang, it won't kill it for you. [22:17] mikeal: because it's EOF + '\n\r' [22:17] isaacs: maushu: for all it knows, you're *trying* to spin the processor for hours. [22:17] _ry: maushu: yes [22:17] mikeal: i think that's why the cohersion is there [22:17] _ry: mikeal: i think === should fix it [22:17] maushu: Ouch. [22:18] mikeal: i think i already did that once [22:18] mikeal: and it broke something else [22:18] mikeal: but i'll do it again [22:18] isaacs: _ry mikeal: you know, you could just set your flag values like EOF and such to objects. then === will always be reliable. [22:18] mikeal: so, you expect this._writeQueueLast() to be an integer 42? [22:18] WALoeIII has joined the channel [22:18] _ry: mikeal: do it also in the other places where END_OF_FILE is compared [22:18] _ry: mikeal: yeah [22:18] mikeal: ok [22:19] maushu: Still, 2mb for each node.js process.... hmmm. [22:19] _ry: maushu: have the node process ping back ever few seconds [22:19] _ry: if it doesn't kill it [22:20] _ry: try to reuse the process as much as possible [22:20] _ry: use unix sockets for IPC [22:20] maushu: _ry, yes, that was my idea too. Keep a tcp connection, if it doesn't answer (and it knows the process is still there) then kill it. [22:20] maushu: unix sockets for local connections, tcp for remote sandboxes. [22:20] maushu: Cloud computing, yay! [22:20] _ry: maushu: then you have a pool of those processes [22:20] _ry: it's very secure [22:21] _ry: of course you need to not load the node libraries [22:21] _ry: which i'd be willing to hack in as a command line flag [22:21] _ry: ./node --bare [22:21] _ry: or something [22:21] maushu: \o/ [22:21] _ry: node --no-io [22:22] maushu: Huh, some libraries would still be needed, like io to communicate to the parent. [22:22] _ry: oh i guess if you're running stuff in a separate context that's not necessary [22:23] Tim_Smart has joined the channel [22:23] gf3 has joined the channel [22:25] maushu: Bye process.mixin. [22:25] maushu: We will miss you. [22:29] devinus has joined the channel [22:29] isaacs: maushu: speak for yourself [22:30] gwoo: hehe [22:30] isaacs: ACTION wasn't really a fan of process.mixin. [22:30] gwoo: good riddens [22:30] maushu: Poor poor mixin. You are so cruel. [22:30] gwoo: haha [22:32] _ry: mikeal: did the === help? [22:32] mikeal: yes [22:32] mikeal: but i'm seeing if there are any other problems [22:32] mikeal: i'm seeing client errors more often now [22:32] mikeal: and I'm trying to figure out if they are related [22:34] mikeal: did remove listener go back to only taking the function? [22:35] siong1987 has joined the channel [22:35] mikeal: oh i see [22:35] mikeal: bad merge on my part [22:36] creationix has joined the channel [22:36] malkomalko has joined the channel [22:44] CIA-77: node: 03Ryan Dahl 07master * r4e7e2f8 10/ (3 files in 2 dirs): [22:44] CIA-77: node: Change nextTick implementation for the better [22:44] CIA-77: node: Use a prepare and idle watcher to execute the nextTick callback more [22:44] CIA-77: node: quickly. Test provided by Matt Ranney. - http://bit.ly/c7Roas [22:44] CIA-77: node: 03Ryan Dahl 07master * ra98d23d 10/ (src/node.cc wscript): Disable IdleWatcher - http://bit.ly/bhkU7W [22:44] _ry: isaacs: would love to see an updated setTimeout(fn, 0) vs nextTick bench :) [22:45] isaacs: _ry: you can just use the existing one. [22:45] isaacs: _ry: the bench code doesn't have to change. [22:45] _ry: ACTION doesn't have node-bench installed though [22:45] _ry: i was scared away by the 'make install' [22:46] isaacs: _ry: then skip that bit. [22:46] isaacs: _ry: it doesn't do much [22:48] _ry: isaacs: i feel like i should be able to specify a node to build with [22:48] isaacs: isaacs: hm. that's a good point. [22:48] isaacs: er... _ry [22:49] isaacs: i should really rejigger it to make better use of npm. that would make it automagically run with whichever node is doing the installing [22:49] isaacs: and be installed into the same dir [22:49] _ry: it'd be nice if i could just do "/my/node bench.js benchscript.js [22:49] mikeal: ok _ry, I've got a new bug now [22:49] mikeal: i'm getting client errors [22:49] mikeal: Error: Parse Error [22:49] isaacs: _ry: i could make that work. [22:51] _ry: isaacs: or better '/my/node bnechscript.js' :) [22:52] isaacs: _ry: sure. maybe then instead of doing a #! in the benchscript, you could do something like require("bench").runMe() [22:52] isaacs: or require("bench").run({...}) [22:53] _ry: mikeal: mmm - malformated http [22:54] brapse has joined the channel [22:54] mikeal: maybe, i doubt it, when i request the same content with a node http request it's fine [22:54] mikeal: pure node that is [22:54] mikeal: this just never ends [22:58] CIA-77: node: 03Ryan Dahl 07master * r71dc232 10/ lib/net.js : [22:58] CIA-77: node: Use === instead of == for END_OF_FILE compares [22:58] CIA-77: node: This caused a very hard to track down bug. Thanks to Mikeal Rogers for this [22:58] CIA-77: node: fix. Unfortunately we were unable to put together a test case. - http://bit.ly/aQ5yy2 [23:00] maushu: For a second there I felt like singing the song that never ends. [23:01] charlesjolley has joined the channel [23:03] RayMorgan_ has joined the channel [23:08] mikeal: _ry: now i'm stepping on this bug http://github.com/ry/node/issues#issue/77 [23:09] _ry: oh geez [23:10] bpot has joined the channel [23:11] hassox has joined the channel [23:14] mjijackson: What is everyone's favorite syntax highlighting library in JS? [23:14] malkomalko has joined the channel [23:14] mjijackson: That is hopefully still being maintained by someone... [23:18] freshtonic has joined the channel [23:18] mjr_: Oh man, that issue 77 is still with us? I thought for sure all of the recent fixes would have cleared that one up. [23:19] _ry: me oto [23:20] maushu: I have this small webserver that sends images. I have this 160kb image that for some reason sometimes the server only sends 128kb. Any reason why? [23:20] maushu: According to my debug lines it reads the whole file, so the problem is probably not there. [23:21] maushu: ps: I'm streaming the file directly to the http response. [23:22] maushu: Wait, that would be, would it? Closing the connection before the rest is sent... [23:22] CIA-77: node: 03Ryan Dahl 07master * r6732e7f 10/ test/pummel/test-http-big-proxy-responses.js : [23:22] CIA-77: node: Add big proxy failing test [23:22] CIA-77: node: GH-77. Code by Robert Newson - http://bit.ly/aN2oZ1 [23:28] emyller has joined the channel [23:28] mikeal: _ry: onMessageComplete calls parser.incoming.emit('end') [23:29] mikeal: is that suppose to bubble up, or is that just an individual message? [23:31] _ry: mikeal: you mean to the connection? no [23:31] mikeal: ok [23:33] _ry: mikeal: there aren't any HEAD requests involved? [23:34] mikeal: there was one earlier, but i use a workaround for that [23:34] mikeal: I'm testing this with the testcase in the bug now [23:36] hassox has joined the channel [23:39] _ry: that test case is debatable [23:39] _ry: acually [23:39] _ry: okay nevermind [23:42] mikeal: you're ok with it now? [23:42] _ry: well - i mean it puts so much into memory [23:42] _ry: it's not really fair [23:42] _ry: there really needs to be some throttling [23:43] mikeal: for what it's worth I'm seeing the same thing with real work traffic [23:43] CIA-77: node: 03Ryan Dahl 07master * rb549b3f 10/ test/pummel/test-http-big-proxy-responses.js : Fix test case style - http://bit.ly/c6DmHJ [23:43] mikeal: hehe, i just actually fixed some of the style independently :) [23:44] _ry: i mean where i loops, writing that response [23:44] _ry: it should really be checking the return value of write [23:44] _ry: and the wait for drain [23:44] mjr_: _ry: is there an easy way to expose the write queue? [23:45] _ry: mjr_: it's a bit tricky [23:45] mjr_: It wouldn't be hard to do it manually if you could tell how much was in the write queue [23:45] mjr_: I've looked a couple of times, and I couldn't figure it out. [23:45] _ry: mikeal: the problem is on http streams write() always returns true [23:45] _ry: and never emits 'drain' [23:45] mikeal: i know [23:46] mikeal: but that's not the problem with this bug :) [23:46] mikeal: maybe this test [23:46] mikeal: but I'm seeing this with normal traffic on a large _changes feed [23:46] _ry: maybe - but this isn't recommended behavior - storing hundreds of MB in memory [23:46] _ry: you wouldn't if you had throttling [23:47] _ry: what i'm saying is - there are two bugs :) [23:48] _ry: http doesn't have throttling - and something is fucked up with extremely large buggers [23:48] _ry: buffers [23:48] _ry: maybe i shoud try to do the http throttling right now [23:51] _ry: http throttling + http client pipelining + client pool abstraction = my dream [23:53] mikeal: if you get the client object to do keepalive by default [23:54] mikeal: i'll write a client pool abstraction :) [23:54] mikeal: ok, i gotta run [23:55] mikeal: wish i had the C chops to dig in to the parser [23:56] mikeal: :( [23:57] CIA-77: node: 03Ryan Dahl 07master * ra934566 10/ (2 files in 2 dirs): [23:57] CIA-77: node: Disable test-idle-watcher [23:57] CIA-77: node: IdleWatcher was disabled in a98d23d9058cdf71a452e71bf8831e4ba2445b50 - http://bit.ly/9Qb98z [23:58] _ry: i'm fairly positive the parser isn't broke [23:58] _ry: (modulo HEAD)