Comments for Christopher Warner http://people.kernelcode.com/cwarner "So, it's not like the code is my brain; keep it" Tue, 06 Jan 2009 14:49:44 +0000 http://wordpress.org/?v=2.6.1 Comment on Plone, so where is it useful? - EXAMPLE #1 by Christopher Warner http://people.kernelcode.com/cwarner/2008/12/11/plone-so-where-is-it-useful-example-1/#comment-6862 Christopher Warner Fri, 19 Dec 2008 20:33:23 +0000 http://people.kernelcode.com/cwarner/?p=139#comment-6862 Yeah I'd need to know more about Endeca and how it works but basically what i'm talking about is what it does before that data is presented to Endeca. What Endeca does with the data I'm unconcerned with. So how does the data get into Endeca? If it's running a cron job that scans the entire filesystem all of the time; We agree, that's pretty silly :-) Yeah I’d need to know more about Endeca and how it works but basically what i’m talking about is what it does before that data is presented to Endeca. What Endeca does with the data I’m unconcerned with.

So how does the data get into Endeca? If it’s running a cron job that scans the entire filesystem all of the time; We agree, that’s pretty silly :-)

]]> Comment on Plone, so where is it useful? - EXAMPLE #1 by developer_girl http://people.kernelcode.com/cwarner/2008/12/11/plone-so-where-is-it-useful-example-1/#comment-6861 developer_girl Fri, 19 Dec 2008 20:12:02 +0000 http://people.kernelcode.com/cwarner/?p=139#comment-6861 The way I understood your post is you can't use the event system you only wish you could. I agree if its built in then definitely use it, but thats not the case. You would actually want to run this job in the endeca pipeline, so it runs the dump before it needs it rather than in cron. Try dumping the whole XML file and at the same time run top on your machine. Look at the resource usage. The way I understood your post is you can’t use the event system you only wish you could. I agree if its built in then definitely use it, but thats not the case. You would actually want to run this job in the endeca pipeline, so it runs the dump before it needs it rather than in cron. Try dumping the whole XML file and at the same time run top on your machine. Look at the resource usage.

]]> Comment on Plone, so where is it useful? - EXAMPLE #1 by Christopher Warner http://people.kernelcode.com/cwarner/2008/12/11/plone-so-where-is-it-useful-example-1/#comment-6860 Christopher Warner Fri, 19 Dec 2008 17:48:02 +0000 http://people.kernelcode.com/cwarner/?p=139#comment-6860 Also note that when I say my way I mean using the event system under Plone.. Inotify is under the work harder way.. It would still be faster and more simple than the crob job though. So it'd be more like driving a car to the corner store. So what is more simple than.. When an event occurs like say; XThisHasChanged we kick off a XThingHasChangedEvent and do whatever we would do as we need to do it. It doesn't get anymore simple than that. Cron jobs aren't a proper replacement for an event system. When something happens; do something. ORRRRR your way where we don't know when something happens, so just run this job over here, over; "Nevermind me running back and forth doing the same thing over and over again on the same stuff, even though nothing has changed!" Any Plonistas or Kernel ppl wanna comment? Also note that when I say my way I mean using the event system under Plone.. Inotify is under the work harder way.. It would still be faster and more simple than the crob job though. So it’d be more like driving a car to the corner store.

So what is more simple than.. When an event occurs like say; XThisHasChanged we kick off a XThingHasChangedEvent and do whatever we would do as we need to do it. It doesn’t get anymore simple than that. Cron jobs aren’t a proper replacement for an event system.

When something happens; do something. ORRRRR your way where we don’t know when something happens, so just run this job over here, over; “Nevermind me running back and forth doing the same thing over and over again on the same stuff, even though nothing has changed!”

Any Plonistas or Kernel ppl wanna comment?

]]> Comment on Plone, so where is it useful? - EXAMPLE #1 by Christopher Warner http://people.kernelcode.com/cwarner/2008/12/11/plone-so-where-is-it-useful-example-1/#comment-6858 Christopher Warner Fri, 19 Dec 2008 17:31:00 +0000 http://people.kernelcode.com/cwarner/?p=139#comment-6858 Hi, besides the fact that inotify was created exactly for this purpose "Its major use is therefore arguably in desktop search utilities like Beagle, where its functionality permits reindexing of changed files without scanning the filesystem for changes every few minutes, which would be very inefficient." via http://en.wikipedia.org/wiki/Inotify The problem is that KISS applies to something engineered and designed well ahead of time. If you want; we can count the amount of system calls you would use; and the amount of system calls I would use. Then ontop of that it would take sufficiently LESS time to implement. I don't know much about endeca partial updates or endeca in general but it's a userspace application and most likely NOT using inotify. Inotify was put in the kernel to replace dnotify specifically because for large ops it would be quicker. Another overlook is that I have NFS in the mix polling that mount or running cron jobs on a mount that is already heavily overworked especially in my case is slow. Anyway, using another database isn't really the problem.. The problem is if I did it your way.. I would be flying a plane to go to pick up some oj from the corner store. I agree that your way would work, but I don't really feel like flying an airplane and going all the way around the world to pick up my oj. Your way is only KISS when you don't have to worry about the system and it's resources (ie: pilot of plane). Don't be a sloppy developer. You could just WALK to the corner store? Work SMARTER not HARDER. Hi, besides the fact that inotify was created exactly for this purpose

“Its major use is therefore arguably in desktop search utilities like Beagle, where its functionality permits reindexing of changed files without scanning the filesystem for changes every few minutes, which would be very inefficient.” via http://en.wikipedia.org/wiki/Inotify

The problem is that KISS applies to something engineered and designed well ahead of time. If you want; we can count the amount of system calls you would use; and the amount of system calls I would use. Then ontop of that it would take sufficiently LESS time to implement. I don’t know much about endeca partial updates or endeca in general but it’s a userspace application and most likely NOT using inotify. Inotify was put in the kernel to replace dnotify specifically because for large ops it would be quicker. Another overlook is that I have NFS in the mix polling that mount or running cron jobs on a mount that is already heavily overworked especially in my case is slow. Anyway, using another database isn’t really the problem.. The problem is if I did it your way.. I would be flying a plane to go to pick up some oj from the corner store. I agree that your way would work, but I don’t really feel like flying an airplane and going all the way around the world to pick up my oj. Your way is only KISS when you don’t have to worry about the system and it’s resources (ie: pilot of plane). Don’t be a sloppy developer. You could just WALK to the corner store?

Work SMARTER not HARDER.

]]> Comment on Plone, so where is it useful? - EXAMPLE #1 by developer_girl http://people.kernelcode.com/cwarner/2008/12/11/plone-so-where-is-it-useful-example-1/#comment-6857 developer_girl Fri, 19 Dec 2008 10:02:50 +0000 http://people.kernelcode.com/cwarner/?p=139#comment-6857 Hi, I stumbled on this blog looking for endeca support. I have to disagree with this. I would use the cron job. The number one concern for developers is managing complexity. By adding the code (or using a tool like inotify) to check for updates and opening the file and updating you add a significant amount of complexity to this process. I don't know how mission critical your application is or the running time of the code to process the content and turn it into XML, but with computers these days most applications take less than a couple minutes to run. Your endeca partial updates are probably not running every minute so you can set something more reasonable for the time on the cron job. If you aren't using a production server database (you shouldn't!) for these dumps then the small amount of overhead it add is insignificant. And if your databases can fit in ram then its negligible. KISS unless you are REALLY observing performance issues. http://en.wikipedia.org/wiki/KISS_principle Hi, I stumbled on this blog looking for endeca support. I have to disagree with this. I would use the cron job. The number one concern for developers is managing complexity. By adding the code (or using a tool like inotify) to check for updates and opening the file and updating you add a significant amount of complexity to this process. I don’t know how mission critical your application is or the running time of the code to process the content and turn it into XML, but with computers these days most applications take less than a couple minutes to run. Your endeca partial updates are probably not running every minute so you can set something more reasonable for the time on the cron job. If you aren’t using a production server database (you shouldn’t!) for these dumps then the small amount of overhead it add is insignificant. And if your databases can fit in ram then its negligible.

KISS unless you are REALLY observing performance issues.
http://en.wikipedia.org/wiki/KISS_principle

]]>