From lists at thekerns.net Thu Jun 2 10:37:16 2005 From: lists at thekerns.net (Matt Kern) Date: Thu, 2 Jun 2005 10:37:16 -0400 Subject: [atlphp-talk] Headcount for tonight's meeting? Message-ID: <4B811CB2-1CB7-415E-BC04-322C3107327E@thekerns.net> Please respond to the list if your planning on attending this evening. I'm trying to get a headcount to make sure the room is big enough. See you tonight! Matt Kern --------------------------------------------------------------------- Atlanta PHP Talk http://www.atlphp.org/archives/talk http://www.atlphp.org --------------------------------------------------------------------- To unsubscribe, e-mail: talk-unsubscribe at lists.atlphp.org For additional commands, e-mail: talk-help at lists.atlphp.org From ben at atlphp.org Thu Jun 2 10:39:04 2005 From: ben at atlphp.org (Ben Ramsey) Date: Thu, 02 Jun 2005 10:39:04 -0400 Subject: [atlphp-talk] Headcount for tonight's meeting? In-Reply-To: <4B811CB2-1CB7-415E-BC04-322C3107327E@thekerns.net> References: <4B811CB2-1CB7-415E-BC04-322C3107327E@thekerns.net> Message-ID: <429F1A08.7050704@atlphp.org> Matt Kern wrote: > Please respond to the list if your planning on attending this evening. > I'm trying to get a headcount to make sure the room is big enough. I'll be there, of course. :-) -- Ben Ramsey Atlanta PHP http://atlphp.org/ --------------------------------------------------------------------- Atlanta PHP Talk http://www.atlphp.org/archives/talk http://www.atlphp.org --------------------------------------------------------------------- To unsubscribe, e-mail: talk-unsubscribe at lists.atlphp.org For additional commands, e-mail: talk-help at lists.atlphp.org From cbc at hylaweb.com Thu Jun 2 11:08:53 2005 From: cbc at hylaweb.com (Callie B. Carroll) Date: Thu, 2 Jun 2005 11:08:53 -0400 (EDT) Subject: [atlphp-talk] Headcount for tonight's meeting? In-Reply-To: <4B811CB2-1CB7-415E-BC04-322C3107327E@thekerns.net> References: <4B811CB2-1CB7-415E-BC04-322C3107327E@thekerns.net> Message-ID: <56276.204.95.150.205.1117724933.squirrel@webmail.dreamhost.com> I'll be there. > Please respond to the list if your planning on attending this > evening. I'm trying to get a headcount to make sure the room is big > enough. > > See you tonight! > > Matt Kern --------------------------------------------------------------------- Atlanta PHP Talk http://www.atlphp.org/archives/talk http://www.atlphp.org --------------------------------------------------------------------- To unsubscribe, e-mail: talk-unsubscribe at lists.atlphp.org For additional commands, e-mail: talk-help at lists.atlphp.org From lists at neoncowboy.com Thu Jun 2 11:22:34 2005 From: lists at neoncowboy.com (John Corry) Date: Thu, 2 Jun 2005 11:22:34 -0400 Subject: [atlphp-talk] Headcount for tonight's meeting? In-Reply-To: <56276.204.95.150.205.1117724933.squirrel@webmail.dreamhost.com> Message-ID: I *may* be able to attend...and I may wind up hanging out at home with our daughter so the wifey can get out. John > -----Original Message----- > From: Callie B. Carroll [mailto:cbc at hylaweb.com] > Sent: Thursday, June 02, 2005 11:09 AM > To: talk at lists.atlphp.org > Subject: Re: [atlphp-talk] Headcount for tonight's meeting? > > I'll be there. > > > Please respond to the list if your planning on attending > this evening. > > I'm trying to get a headcount to make sure the room is big enough. > > > > See you tonight! > > > > Matt Kern > > > --------------------------------------------------------------------- > Atlanta PHP Talk > http://www.atlphp.org/archives/talk > http://www.atlphp.org > --------------------------------------------------------------------- > To unsubscribe, e-mail: talk-unsubscribe at lists.atlphp.org For > additional commands, e-mail: talk-help at lists.atlphp.org > > --------------------------------------------------------------------- Atlanta PHP Talk http://www.atlphp.org/archives/talk http://www.atlphp.org --------------------------------------------------------------------- To unsubscribe, e-mail: talk-unsubscribe at lists.atlphp.org For additional commands, e-mail: talk-help at lists.atlphp.org From necrotic at gmail.com Thu Jun 2 11:25:02 2005 From: necrotic at gmail.com (James Logsdon) Date: Thu, 02 Jun 2005 15:25:02 -0000 Subject: [atlphp-talk] Headcount for tonight's meeting? In-Reply-To: <4B811CB2-1CB7-415E-BC04-322C3107327E@thekerns.net> References: <4B811CB2-1CB7-415E-BC04-322C3107327E@thekerns.net> Message-ID: On Thu, 02 Jun 2005 14:37:16 -0000, Matt Kern wrote: > Please respond to the list if your planning on attending this evening. > I'm trying to get a headcount to make sure the room is big enough. > > See you tonight! > > Matt Kern > > --------------------------------------------------------------------- > Atlanta PHP Talk > http://www.atlphp.org/archives/talk > http://www.atlphp.org > --------------------------------------------------------------------- > To unsubscribe, e-mail: talk-unsubscribe at lists.atlphp.org > For additional commands, e-mail: talk-help at lists.atlphp.org > I'll be there :) --------------------------------------------------------------------- Atlanta PHP Talk http://www.atlphp.org/archives/talk http://www.atlphp.org --------------------------------------------------------------------- To unsubscribe, e-mail: talk-unsubscribe at lists.atlphp.org For additional commands, e-mail: talk-help at lists.atlphp.org From john at neoncowboy.com Thu Jun 2 11:27:40 2005 From: john at neoncowboy.com (John Corry) Date: Thu, 2 Jun 2005 11:27:40 -0400 Subject: [atlphp-talk] Php table parsing class Message-ID: I am working on a mini project that will include grabbing some data from external web pages...web pages that are maintained by the government. It would be cool if I could get the state's webmaster to just give me a URL to CSV data, but they won't respond to emails. So, I need to extract a bunch of data from a table. Does anyone know of a class/script/tool that would make this easier? I totally suck at regular expressions... John --------------------------------------------------------------------- Atlanta PHP Talk http://www.atlphp.org/archives/talk http://www.atlphp.org --------------------------------------------------------------------- To unsubscribe, e-mail: talk-unsubscribe at lists.atlphp.org For additional commands, e-mail: talk-help at lists.atlphp.org From ben at atlphp.org Thu Jun 2 11:35:36 2005 From: ben at atlphp.org (Ben Ramsey) Date: Thu, 02 Jun 2005 11:35:36 -0400 Subject: [atlphp-talk] Php table parsing class Message-ID: <429F2748.1040901@atlphp.org> John Corry wrote: > I am working on a mini project that will include grabbing some data from > external web pages...web pages that are maintained by the government. It > would be cool if I could get the state's webmaster to just give me a URL to > CSV data, but they won't respond to emails. > > So, I need to extract a bunch of data from a table. > > Does anyone know of a class/script/tool that would make this easier? I > totally suck at regular expressions... I don't know of any tools that will aid you in doing this, but you can use a combination of file_get_contents() (http://www.php.net/file_get_contents), if using PHP 5, or the cURL functions (http://www.php.net/curl), if using PHP 4, and preg_match() (http://www.php.net/preg_match) to achieve your intended result. This, by the way, is called "screen scraping" and tends to be looked down upon. A polite approach would be to cache the data you retrieve so that you're only accessing the other site once or twice a day (at most, once an hour). -- Ben Ramsey Atlanta PHP http://atlphp.org/ --------------------------------------------------------------------- Atlanta PHP Talk http://www.atlphp.org/archives/talk http://www.atlphp.org --------------------------------------------------------------------- To unsubscribe, e-mail: talk-unsubscribe at lists.atlphp.org For additional commands, e-mail: talk-help at lists.atlphp.org From tulodieski at bellsouth.net Thu Jun 2 11:42:52 2005 From: tulodieski at bellsouth.net (Don Tulodieski) Date: Thu, 02 Jun 2005 11:42:52 -0400 Subject: [atlphp-talk] Php table parsing class In-Reply-To: <20050602152752.WGQ29846.ibm28aec.bellsouth.net@mail.atlphp .org> References: <20050602152752.WGQ29846.ibm28aec.bellsouth.net@mail.atlphp.org> Message-ID: <6.1.0.6.2.20050602113444.01cbb2f8@mail.bellsouth.net> You may be barking up the wrong tree trying with the webmaster. I've had decent luck contacting the agencies themself (by phone), who own/maintain the needed data. Especially if the agency is already making the data publically available in a different form. -Don At 6/2/2005 11:27 AM, you wrote: >I am working on a mini project that will include grabbing some data from >external web pages...web pages that are maintained by the government. It >would be cool if I could get the state's webmaster to just give me a URL to >CSV data, but they won't respond to emails. > >So, I need to extract a bunch of data from a table. > >Does anyone know of a class/script/tool that would make this easier? I >totally suck at regular expressions... > >John > > > >--------------------------------------------------------------------- >Atlanta PHP Talk >http://www.atlphp.org/archives/talk >http://www.atlphp.org >--------------------------------------------------------------------- >To unsubscribe, e-mail: talk-unsubscribe at lists.atlphp.org >For additional commands, e-mail: talk-help at lists.atlphp.org --------------------------------------------------------------------- Atlanta PHP Talk http://www.atlphp.org/archives/talk http://www.atlphp.org --------------------------------------------------------------------- To unsubscribe, e-mail: talk-unsubscribe at lists.atlphp.org For additional commands, e-mail: talk-help at lists.atlphp.org From lists at neoncowboy.com Thu Jun 2 11:45:39 2005 From: lists at neoncowboy.com (John Corry) Date: Thu, 2 Jun 2005 11:45:39 -0400 Subject: [atlphp-talk] Php table parsing class In-Reply-To: <429F2748.1040901@atlphp.org> Message-ID: > I don't know of any tools that will aid you in doing this, > but you can use a combination of file_get_contents() > (http://www.php.net/file_get_contents), if using PHP 5, or > the cURL functions (http://www.php.net/curl), if using PHP 4, > and preg_match() > (http://www.php.net/preg_match) to achieve your intended result. > > This, by the way, is called "screen scraping" and tends to be > looked down upon. A polite approach would be to cache the > data you retrieve so that you're only accessing the other > site once or twice a day (at most, once an hour). It's *sort of* screen-scraping... I am harvesting the GBI's sex-offender registry data to do more interesting stuff with it than they do. My application will, in fact, cache the data (actually, I'm going to store it in a database) and a cron job will harvest the data weekly (on Sunday night, at 3am). The cURL functions are a piece of cake...it's separating out the desired data that I am looking for a little help with. No big deal if I have to brute force it with regexp...but it would be nice if someone knew of a class to parse HTML tables to extract their data. Thanks, John Corry --------------------------------------------------------------------- Atlanta PHP Talk http://www.atlphp.org/archives/talk http://www.atlphp.org --------------------------------------------------------------------- To unsubscribe, e-mail: talk-unsubscribe at lists.atlphp.org For additional commands, e-mail: talk-help at lists.atlphp.org From scott at crisscott.com Thu Jun 2 11:50:26 2005 From: scott at crisscott.com (Scott Mattocks) Date: Thu, 02 Jun 2005 11:50:26 -0400 Subject: [atlphp-talk] Php table parsing class In-Reply-To: <0MKv6A-1DdrtG0FfO-0002O3@mx.perfora.net> References: <0MKv6A-1DdrtG0FfO-0002O3@mx.perfora.net> Message-ID: <429F2AC2.9070104@crisscott.com> John Corry wrote: > The cURL functions are a piece of cake...it's separating out the desired > data that I am looking for a little help with. No big deal if I have to > brute force it with regexp...but it would be nice if someone knew of a class > to parse HTML tables to extract their data. What about treating it as XML? Then you can use one of the nifty XML extension like SimpleXML or DOM and get all of the element contents. If that doesn't work, I would check PEAR. -- Scott Mattocks scott at crisscott.com http://www.crisscott.com http://pear.php.net/user/scottmattocks --------------------------------------------------------------------- Atlanta PHP Talk http://www.atlphp.org/archives/talk http://www.atlphp.org --------------------------------------------------------------------- To unsubscribe, e-mail: talk-unsubscribe at lists.atlphp.org For additional commands, e-mail: talk-help at lists.atlphp.org From ben at atlphp.org Thu Jun 2 11:52:32 2005 From: ben at atlphp.org (Ben Ramsey) Date: Thu, 02 Jun 2005 11:52:32 -0400 Subject: [atlphp-talk] Php table parsing class Message-ID: <429F2B40.7020603@atlphp.org> John Corry wrote: > It's *sort of* screen-scraping... How is it only "sort of" screen-scraping? This is exactly what screen scraping is. http://en.wikipedia.org/wiki/Screen_scraping :-) Not that I have anything against it. I've done it plenty of times. > I am harvesting the GBI's sex-offender registry data to do more interesting > stuff with it than they do. > > My application will, in fact, cache the data (actually, I'm going to store > it in a database) and a cron job will harvest the data weekly (on Sunday > night, at 3am). > > The cURL functions are a piece of cake...it's separating out the desired > data that I am looking for a little help with. No big deal if I have to > brute force it with regexp...but it would be nice if someone knew of a class > to parse HTML tables to extract their data. You may also want to look at the Tidy functions (http://en.wikipedia.org/wiki/Screen_scraping) in PHP, which will help you to validate the HTML and traverse the document tree. You could probably also do something similar with SimpleXML (http://us3.php.net/simplexml), which requires PHP 5. I guess the approach would be to use Tidy to ensure that the HTML is valid (Tidy will not only validate it, but it will close all open-ended tags and clean it up), then use SimpleXML to parse the document as if it's XML (since you can do that with well-formed HTML/XHTML). Then, with a proper XML tree using SimpleXML, you can get the data quickly without the need for regular expressions. Your regular expressions will also break everytime they change the layout of their page. The SimpleXML approach will not break, though the location of the elements may change. -- Ben Ramsey Atlanta PHP http://atlphp.org/ --------------------------------------------------------------------- Atlanta PHP Talk http://www.atlphp.org/archives/talk http://www.atlphp.org --------------------------------------------------------------------- To unsubscribe, e-mail: talk-unsubscribe at lists.atlphp.org For additional commands, e-mail: talk-help at lists.atlphp.org From lists at neoncowboy.com Thu Jun 2 12:00:52 2005 From: lists at neoncowboy.com (John Corry) Date: Thu, 2 Jun 2005 12:00:52 -0400 Subject: [atlphp-talk] Php table parsing class In-Reply-To: <429F2B40.7020603@atlphp.org> Message-ID: Looks like I'm getting warmer. I don't have PHP 5 yet...so simpleXML is out. When your message came in I was reading about the Tidy functions. I said it was 'sort of' screen scraping since I'm only hitting their site/harvesting their data once a week...not every time I get a request. I guess according to the definition...that's what I'm doing. I'm not going to lose any sleep over it though since: A) mine and my audience's tax $$ paid for the screen that's being scraped B) I'm doing better stuff with the data than the GBI is :) Back to tidy functions... Thanks, John > -----Original Message----- > From: Ben Ramsey [mailto:ben at atlphp.org] > Sent: Thursday, June 02, 2005 11:53 AM > To: talk at lists.atlphp.org > Subject: Re: [atlphp-talk] Php table parsing class > > John Corry wrote: > > It's *sort of* screen-scraping... > > How is it only "sort of" screen-scraping? This is exactly > what screen scraping is. http://en.wikipedia.org/wiki/Screen_scraping > > :-) Not that I have anything against it. I've done it plenty of times. > > > I am harvesting the GBI's sex-offender registry data to do more > > interesting stuff with it than they do. > > > > My application will, in fact, cache the data (actually, I'm > going to > > store it in a database) and a cron job will harvest the data weekly > > (on Sunday night, at 3am). > > > > The cURL functions are a piece of cake...it's separating out the > > desired data that I am looking for a little help with. No > big deal if > > I have to brute force it with regexp...but it would be nice > if someone > > knew of a class to parse HTML tables to extract their data. > > You may also want to look at the Tidy functions > (http://en.wikipedia.org/wiki/Screen_scraping) in PHP, which > will help you to validate the HTML and traverse the document > tree. You could probably also do something similar with > SimpleXML (http://us3.php.net/simplexml), which requires PHP > 5. I guess the approach would be to use Tidy to ensure that > the HTML is valid (Tidy will not only validate it, but it > will close all open-ended tags and clean it up), then use > SimpleXML to parse the document as if it's XML (since you can > do that with well-formed HTML/XHTML). > > Then, with a proper XML tree using SimpleXML, you can get the > data quickly without the need for regular expressions. Your > regular expressions will also break everytime they change the > layout of their page. The SimpleXML approach will not break, > though the location of the elements may change. > > -- > Ben Ramsey > Atlanta PHP > http://atlphp.org/ > > --------------------------------------------------------------------- > Atlanta PHP Talk > http://www.atlphp.org/archives/talk > http://www.atlphp.org > --------------------------------------------------------------------- > To unsubscribe, e-mail: talk-unsubscribe at lists.atlphp.org > For additional commands, e-mail: talk-help at lists.atlphp.org > > --------------------------------------------------------------------- Atlanta PHP Talk http://www.atlphp.org/archives/talk http://www.atlphp.org --------------------------------------------------------------------- To unsubscribe, e-mail: talk-unsubscribe at lists.atlphp.org For additional commands, e-mail: talk-help at lists.atlphp.org From scott at crisscott.com Thu Jun 2 12:12:18 2005 From: scott at crisscott.com (Scott Mattocks) Date: Thu, 02 Jun 2005 12:12:18 -0400 Subject: [atlphp-talk] Php table parsing class In-Reply-To: <0MKuxu-1Dds800vnV-0007Lk@mx.perfora.net> References: <0MKuxu-1Dds800vnV-0007Lk@mx.perfora.net> Message-ID: <429F2FE2.3070606@crisscott.com> John Corry wrote: > I don't have PHP 5 yet...so simpleXML is out. When your message came in I > was reading about the Tidy functions. You might want to look at PEAR::XML_HTMLSax3 which is "A SAX parser for HTML and other badly formed XML documents" http://pear.php.net/package/XML_HTMLSax3 -- Scott Mattocks scott at crisscott.com http://www.crisscott.com http://pear.php.net/user/scottmattocks --------------------------------------------------------------------- Atlanta PHP Talk http://www.atlphp.org/archives/talk http://www.atlphp.org --------------------------------------------------------------------- To unsubscribe, e-mail: talk-unsubscribe at lists.atlphp.org For additional commands, e-mail: talk-help at lists.atlphp.org From ben at atlphp.org Thu Jun 2 12:28:55 2005 From: ben at atlphp.org (Ben Ramsey) Date: Thu, 02 Jun 2005 12:28:55 -0400 Subject: [atlphp-talk] Php table parsing class Message-ID: <429F33C7.4080505@atlphp.org> John Corry wrote: > I'm not going to lose any sleep over it though since: > A) mine and my audience's tax $$ paid for the screen that's being scraped > B) I'm doing better stuff with the data than the GBI is LOL! I hear you. ;-) Good luck! -- Ben Ramsey Atlanta PHP http://atlphp.org/ --------------------------------------------------------------------- Atlanta PHP Talk http://www.atlphp.org/archives/talk http://www.atlphp.org --------------------------------------------------------------------- To unsubscribe, e-mail: talk-unsubscribe at lists.atlphp.org For additional commands, e-mail: talk-help at lists.atlphp.org From sprocket at f2o.org Thu Jun 2 12:55:10 2005 From: sprocket at f2o.org (Chris Spruck) Date: Thu, 2 Jun 2005 12:55:10 -0400 (EDT) Subject: [atlphp-talk] Headcount for tonight's meeting? In-Reply-To: <4B811CB2-1CB7-415E-BC04-322C3107327E@thekerns.net> References: <4B811CB2-1CB7-415E-BC04-322C3107327E@thekerns.net> Message-ID: <4274.4.154.88.172.1117731310.squirrel@www.f2o.org> I'll be there. Chris --------------------------------------------------------------------- Atlanta PHP Talk http://www.atlphp.org/archives/talk http://www.atlphp.org --------------------------------------------------------------------- To unsubscribe, e-mail: talk-unsubscribe at lists.atlphp.org For additional commands, e-mail: talk-help at lists.atlphp.org From necrotic at gmail.com Thu Jun 2 19:56:38 2005 From: necrotic at gmail.com (James Logsdon) Date: Thu, 02 Jun 2005 19:56:38 -0400 Subject: [atlphp-talk] Great Meeting Message-ID: Hey, I enjoyed the meeting tonight. Ben's talk was informative. I knew a little about what he had to say, but whatever questions I had before, he answered with his presentation. The discussion aftewards was good, too, in my opinion. Matt said he would probably present in the July meeting. I'll let him post up his topics he talked about so I don't miss anything. I'm going to go ahead and volunteer to do the August presentation. I've got something in mind that I've already started a little work on. My original topic was "Enterprise Frameworks and CMSes and the Open-Source World." I've decided to broaden it to more of an "Open-Source meets Enterprise" sort of presentation. I found a lot of stuff when researching the first topic, and the first one was way to narrow. Signed, James Logsdon --------------------------------------------------------------------- Atlanta PHP Talk http://www.atlphp.org/archives/talk http://www.atlphp.org --------------------------------------------------------------------- To unsubscribe, e-mail: talk-unsubscribe at lists.atlphp.org For additional commands, e-mail: talk-help at lists.atlphp.org From lists at neoncowboy.com Fri Jun 3 07:37:28 2005 From: lists at neoncowboy.com (John Corry) Date: Fri, 3 Jun 2005 07:37:28 -0400 Subject: [atlphp-talk] Great Meeting In-Reply-To: Message-ID: Ditto! It was great to meet you all and great to be around other folks who do the same thing. Ben, your presentation was terrific...great job. I was very relieved to learn that most of the stuff you talked about is coding practices I already adhere to (even though I didn't know specifically what I was guarding against) and that the other stuff I avoid by being the only user with access to my server . James: I go through Lawrenceville...if you ever want to carpool or anything. See y'all next month! John > -----Original Message----- > From: James Logsdon [mailto:necrotic at gmail.com] > Sent: Thursday, June 02, 2005 7:57 PM > To: AtlPHP.Talk > Subject: [atlphp-talk] Great Meeting > > Hey, > > I enjoyed the meeting tonight. Ben's talk was informative. I > knew a little about what he had to say, but whatever > questions I had before, he answered with his presentation. > The discussion aftewards was good, too, in my opinion. > > Matt said he would probably present in the July meeting. I'll > let him post up his topics he talked about so I don't miss anything. > > I'm going to go ahead and volunteer to do the August > presentation. I've got something in mind that I've already > started a little work on. My original topic was "Enterprise > Frameworks and CMSes and the Open-Source World." I've decided > to broaden it to more of an "Open-Source meets Enterprise" > sort of presentation. I found a lot of stuff when researching > the first topic, and the first one was way to narrow. > > Signed, > James Logsdon > > --------------------------------------------------------------------- > Atlanta PHP Talk > http://www.atlphp.org/archives/talk > http://www.atlphp.org > --------------------------------------------------------------------- > To unsubscribe, e-mail: talk-unsubscribe at lists.atlphp.org For > additional commands, e-mail: talk-help at lists.atlphp.org > > --------------------------------------------------------------------- Atlanta PHP Talk http://www.atlphp.org/archives/talk http://www.atlphp.org --------------------------------------------------------------------- To unsubscribe, e-mail: talk-unsubscribe at lists.atlphp.org For additional commands, e-mail: talk-help at lists.atlphp.org From pstevereynolds at bellsouth.net Fri Jun 3 09:04:24 2005 From: pstevereynolds at bellsouth.net (Steve Reynolds) Date: Fri, 3 Jun 2005 09:04:24 -0400 Subject: [atlphp-talk] RE: @SPAM++++++ RE: [atlphp-talk] Great Meeting In-Reply-To: <20050603113739.KGXV14990.ibm15aec.bellsouth.net@mail.atlphp.org> Message-ID: <20050603130521.NEHT4468.ibm61aec.bellsouth.net@steve> I agree. Ben, lightbulbs kept going off in my head. I came home and tried a few of the hacks on one of my sites and WHOA!! I got some fixin' to do. (For those of you not from the south, that means I have a good deal of work to perform. :-). Steve Reynolds, President Reynolds Computer Consulting (678) 754-3738 -----Original Message----- From: John Corry [mailto:lists at neoncowboy.com] Sent: Friday, June 03, 2005 7:37 AM To: talk at lists.atlphp.org Subject: @SPAM++++++ RE: [atlphp-talk] Great Meeting Ditto! It was great to meet you all and great to be around other folks who do the same thing. Ben, your presentation was terrific...great job. I was very relieved to learn that most of the stuff you talked about is coding practices I already adhere to (even though I didn't know specifically what I was guarding against) and that the other stuff I avoid by being the only user with access to my server . James: I go through Lawrenceville...if you ever want to carpool or anything. See y'all next month! John > -----Original Message----- > From: James Logsdon [mailto:necrotic at gmail.com] > Sent: Thursday, June 02, 2005 7:57 PM > To: AtlPHP.Talk > Subject: [atlphp-talk] Great Meeting > > Hey, > > I enjoyed the meeting tonight. Ben's talk was informative. I knew a > little about what he had to say, but whatever questions I had before, > he answered with his presentation. > The discussion aftewards was good, too, in my opinion. > > Matt said he would probably present in the July meeting. I'll let him > post up his topics he talked about so I don't miss anything. > > I'm going to go ahead and volunteer to do the August presentation. > I've got something in mind that I've already started a little work on. > My original topic was "Enterprise Frameworks and CMSes and the > Open-Source World." I've decided to broaden it to more of an > "Open-Source meets Enterprise" > sort of presentation. I found a lot of stuff when researching the > first topic, and the first one was way to narrow. > > Signed, > James Logsdon > > --------------------------------------------------------------------- > Atlanta PHP Talk > http://www.atlphp.org/archives/talk > http://www.atlphp.org > --------------------------------------------------------------------- > To unsubscribe, e-mail: talk-unsubscribe at lists.atlphp.org For > additional commands, e-mail: talk-help at lists.atlphp.org > > --------------------------------------------------------------------- Atlanta PHP Talk http://www.atlphp.org/archives/talk http://www.atlphp.org --------------------------------------------------------------------- To unsubscribe, e-mail: talk-unsubscribe at lists.atlphp.org For additional commands, e-mail: talk-help at lists.atlphp.org -- No virus found in this outgoing message. Checked by AVG Anti-Virus. Version: 7.0.322 / Virus Database: 267.4.1 - Release Date: 6/2/2005 --------------------------------------------------------------------- Atlanta PHP Talk http://www.atlphp.org/archives/talk http://www.atlphp.org --------------------------------------------------------------------- To unsubscribe, e-mail: talk-unsubscribe at lists.atlphp.org For additional commands, e-mail: talk-help at lists.atlphp.org From ben at atlphp.org Fri Jun 3 09:26:38 2005 From: ben at atlphp.org (Ben Ramsey) Date: Fri, 03 Jun 2005 09:26:38 -0400 Subject: [atlphp-talk] RE: @SPAM++++++ RE: [atlphp-talk] Great Meeting In-Reply-To: <20050603130521.NEHT4468.ibm61aec.bellsouth.net@steve> References: <20050603130521.NEHT4468.ibm61aec.bellsouth.net@steve> Message-ID: <42A05A8E.9010603@atlphp.org> Steve Reynolds wrote: > I agree. Ben, lightbulbs kept going off in my head. I came home and tried a > few of the hacks on one of my sites and WHOA!! I got some fixin' to do. (For > those of you not from the south, that means I have a good deal of work to > perform. :-). It's good to hear feedback from everyone. I'm glad y'all enjoyed it. In September or October, I hope to revisit the topic with more focus on the attacks themselves and data filtering/escaping practices. In the meantime, it looks like our July and August meetings are already shaping up with some great topics! -- Ben Ramsey Atlanta PHP http://atlphp.org/ --------------------------------------------------------------------- Atlanta PHP Talk http://www.atlphp.org/archives/talk http://www.atlphp.org --------------------------------------------------------------------- To unsubscribe, e-mail: talk-unsubscribe at lists.atlphp.org For additional commands, e-mail: talk-help at lists.atlphp.org From pstevereynolds at bellsouth.net Fri Jun 3 09:49:56 2005 From: pstevereynolds at bellsouth.net (Steve Reynolds) Date: Fri, 3 Jun 2005 09:49:56 -0400 Subject: [atlphp-talk] Questions about hosting and security for Ben or whomever In-Reply-To: <42A05A8E.9010603@atlphp.org> Message-ID: <20050603135044.UYEX11273.ibm67aec.bellsouth.net@steve> I checked my hosting company (Dreamhost) and they run PHP as CGI scripts. They state that it's more secure since scripts run as the user instead of apache. Any thoughts on that? Steve Reynolds, President Reynolds Computer Consulting (678) 754-3738 -----Original Message----- From: Ben Ramsey [mailto:ben at atlphp.org] Sent: Friday, June 03, 2005 9:27 AM To: talk at lists.atlphp.org Subject: Re: [atlphp-talk] RE: @SPAM++++++ RE: [atlphp-talk] Great Meeting Steve Reynolds wrote: > I agree. Ben, lightbulbs kept going off in my head. I came home and > tried a few of the hacks on one of my sites and WHOA!! I got some > fixin' to do. (For those of you not from the south, that means I have > a good deal of work to perform. :-). It's good to hear feedback from everyone. I'm glad y'all enjoyed it. In September or October, I hope to revisit the topic with more focus on the attacks themselves and data filtering/escaping practices. In the meantime, it looks like our July and August meetings are already shaping up with some great topics! -- Ben Ramsey Atlanta PHP http://atlphp.org/ --------------------------------------------------------------------- Atlanta PHP Talk http://www.atlphp.org/archives/talk http://www.atlphp.org --------------------------------------------------------------------- To unsubscribe, e-mail: talk-unsubscribe at lists.atlphp.org For additional commands, e-mail: talk-help at lists.atlphp.org -- No virus found in this outgoing message. Checked by AVG Anti-Virus. Version: 7.0.322 / Virus Database: 267.4.1 - Release Date: 6/2/2005 --------------------------------------------------------------------- Atlanta PHP Talk http://www.atlphp.org/archives/talk http://www.atlphp.org --------------------------------------------------------------------- To unsubscribe, e-mail: talk-unsubscribe at lists.atlphp.org For additional commands, e-mail: talk-help at lists.atlphp.org From ben at atlphp.org Fri Jun 3 10:27:32 2005 From: ben at atlphp.org (Ben Ramsey) Date: Fri, 03 Jun 2005 10:27:32 -0400 Subject: [atlphp-talk] Questions about hosting and security for Ben or whomever In-Reply-To: <20050603135044.UYEX11273.ibm67aec.bellsouth.net@steve> References: <20050603135044.UYEX11273.ibm67aec.bellsouth.net@steve> Message-ID: <42A068D4.9040002@atlphp.org> Steve Reynolds wrote: > I checked my hosting company (Dreamhost) and they run PHP as CGI scripts. > They state that it's more secure since scripts run as the user instead of > apache. > Any thoughts on that? I can't claim to be an expert on the security of running PHP as CGI vs. running it as an Apache module. Most of the time, people tend to run PHP as a module since it's more seemlessly integrated with Apache and runs faster, so I'm more famliar with running it as an Apache module. It's true that running PHP as an Apache module allows PHP to inherit all the permissions of Apache. I'd like to take a closer look at this (especially since it should be covered in the PHP security book I'm co-authoring). I've been reading through the PHP manual's security section about installing PHP as a CGI binary. If you'd like to follow along, it's located here: http://www.php.net/security.cgi-bin As I learn more about this, I'll post more information here. Actually, I should just create a security resource on the Web site and place this information there. Look for that in the next few weeks. -- Ben Ramsey Atlanta PHP http://atlphp.org/ --------------------------------------------------------------------- Atlanta PHP Talk http://www.atlphp.org/archives/talk http://www.atlphp.org --------------------------------------------------------------------- To unsubscribe, e-mail: talk-unsubscribe at lists.atlphp.org For additional commands, e-mail: talk-help at lists.atlphp.org From matt at atlphp.org Fri Jun 3 10:58:20 2005 From: matt at atlphp.org (Matt Kern) Date: Fri, 3 Jun 2005 10:58:20 -0400 Subject: [atlphp-talk] Questions about hosting and security for Ben or whomever In-Reply-To: <42A068D4.9040002@atlphp.org> References: <20050603135044.UYEX11273.ibm67aec.bellsouth.net@steve> <42A068D4.9040002@atlphp.org> Message-ID: <713DA061-662A-48DB-B48A-02D88D3B1FED@atlphp.org> Performance takes a significant hit on higher traffic websites when PHP is run as CGI. Also there are some authentication schemes that aren't useable with CGI PHP as the PHP executable doesn't have direct access to the request and subrequests within apache (as mod_perl and mod_php4 do). One of the older versions of Dokuwiki relied on this form of authentication, and couldn't be used with CGI. I believe that the CGI will run with the permissions and owner of the file being processed, rather than the permissions of the PHP executable, so it all depends on the security of the file. That said, I'm no expert in this, and I'd encourage more research on the topic. On Jun 3, 2005, at 10:27 AM, Ben Ramsey wrote: > Steve Reynolds wrote: > >> I checked my hosting company (Dreamhost) and they run PHP as CGI >> scripts. >> They state that it's more secure since scripts run as the user >> instead of >> apache. >> Any thoughts on that? >> > > I can't claim to be an expert on the security of running PHP as CGI > vs. running it as an Apache module. Most of the time, people tend > to run PHP as a module since it's more seemlessly integrated with > Apache and runs faster, so I'm more famliar with running it as an > Apache module. It's true that running PHP as an Apache module > allows PHP to inherit all the permissions of Apache. > > I'd like to take a closer look at this (especially since it should > be covered in the PHP security book I'm co-authoring). I've been > reading through the PHP manual's security section about installing > PHP as a CGI binary. If you'd like to follow along, it's located > here: http://www.php.net/security.cgi-bin > > As I learn more about this, I'll post more information here. > Actually, I should just create a security resource on the Web site > and place this information there. Look for that in the next few weeks. Ben, that's a great idea. Let's do that. > > -- > Ben Ramsey > Atlanta PHP > http://atlphp.org/ > > --------------------------------------------------------------------- > Atlanta PHP Talk > http://www.atlphp.org/archives/talk > http://www.atlphp.org > --------------------------------------------------------------------- > To unsubscribe, e-mail: talk-unsubscribe at lists.atlphp.org > For additional commands, e-mail: talk-help at lists.atlphp.org > > Matt Kern --------------------------------------------------------------------- Atlanta PHP Talk http://www.atlphp.org/archives/talk http://www.atlphp.org --------------------------------------------------------------------- To unsubscribe, e-mail: talk-unsubscribe at lists.atlphp.org For additional commands, e-mail: talk-help at lists.atlphp.org From john at johnpeace.org Fri Jun 3 11:43:20 2005 From: john at johnpeace.org (John Corry) Date: Fri, 3 Jun 2005 11:43:20 -0400 Subject: [atlphp-talk] Invalid SQL Message-ID: /hangs head shamefully I know I am overlooking something so stupidly simple that it's embarassing to even bring it up.. ..unfortunately, asking you guys to look at it is the only way I can see getting past this hurdle. I have a page that generates an SQL insert, then runs it. That page generates an error: Database error: Invalid SQL: INSERT INTO offenders ( id , name , address, county ) VALUES (5755, "CORDERO, ROBERT", "6348 GOLDEN ISLES WEST , BAXLEY , GA 31513", "Appling"); INSERT INTO offenders ( id , name , address, county ) VALUES (5560, "DASHER, ALEXANDER JR", "179 WEST ALLEN ST , BAXLEY , GA 31513", "Appling"); INSERT INTO offenders ( id , name , address, county ) VALUES (4876, "GARCIA, MOSES", "662 GARDNER RD LOT 5 , BAXLEY , GA 31513", "Appling"); INSERT INTO offenders ( id , name , address, county ) VALUES (4835, "GASKINS, JIMMIE ALLEN", "894 FOSTER STEADHAM RD , BAXLEY , GA 31513", "Appling"); INSERT INTO offenders ( id , name , address, county ) VALUES (3216, "LORD, JIMMY RAY", "169 EDG ERTON STREET , BAXLEY , GA 31513", "Appling"); INSERT INTO offenders ( id , name , address, county ) VALUES (3016, "MATHIS, WILLIE EUGENE", "54 MIDWAY CHURCH RD , BAXLEY , GA 31513", "Appling"); INSERT INTO offenders ( id , name , address, county ) VALUES (2203, "PERRY, MATTHEW LAVERNE", "500 CENTRAL STREET , BAXLEY , GA 31513", "Appling"); INSERT INTO offenders ( id , name , address, county ) VALUES (1798, "ROBERSON, LARRY B", "528 QUAIL HOLLOW ROAD , BAXLEY , GA 31513", "Appling"); MySQL Error: 1064 (You have an error in your SQL syntax near '; INSERT INTO offenders ( id , name , address, county ) VALUES (5560, "DASHER, A' at line 1) The only problem is...it's not invalid SQL! I can copy the sql (either from the error statement or from echoing the sql onto the page) and paste it into the query window on phpMyAdmin...and it runs. WTF? The table structure is as follows: CREATE TABLE `offenders` ( `id` int(11) NOT NULL default '0', `name` varchar(64) NOT NULL default '', `address` text NOT NULL, `county` varchar(36) NOT NULL default '' ) TYPE=MyISAM; I don't get it...this is the way I always do it: Loop through results Appending insert statements to sql, separated by ';' Once the sql statement is built exit the loop Run the sql query The query runs in phpMyAdmin...but mysql won't run the query from the page. Any ideas why? John --------------------------------------------------------------------- Atlanta PHP Talk http://www.atlphp.org/archives/talk http://www.atlphp.org --------------------------------------------------------------------- To unsubscribe, e-mail: talk-unsubscribe at lists.atlphp.org For additional commands, e-mail: talk-help at lists.atlphp.org From scott at crisscott.com Fri Jun 3 11:49:31 2005 From: scott at crisscott.com (Scott Mattocks) Date: Fri, 03 Jun 2005 11:49:31 -0400 Subject: [atlphp-talk] Invalid SQL In-Reply-To: <0MKuxu-1DeEKY171d-0005Wb@mx.perfora.net> References: <0MKuxu-1DeEKY171d-0005Wb@mx.perfora.net> Message-ID: <42A07C0B.8010203@crisscott.com> John Corry wrote: > I have a page that generates an SQL insert, then runs it. That page > generates an error: > Database error: Invalid SQL: INSERT INTO offenders ( id , name , address, > county ) VALUES (5755, "CORDERO, ROBERT", "6348 GOLDEN ISLES WEST , BAXLEY , > GA 31513", "Appling"); INSERT INTO offenders ( id , name , address, county ) > VALUES (5560, "DASHER, ALEXANDER JR", "179 WEST ALLEN ST , BAXLEY , GA > 31513", "Appling"); INSERT INTO offenders ( id , name , address, county ) > VALUES (4876, "GARCIA, MOSES", "662 GARDNER RD LOT 5 , BAXLEY , GA 31513", > "Appling"); INSERT INTO offenders ( id , name , address, county ) VALUES > (4835, "GASKINS, JIMMIE ALLEN", "894 FOSTER STEADHAM RD , BAXLEY , GA > 31513", "Appling"); INSERT INTO offenders ( id , name , address, county ) > VALUES (3216, "LORD, JIMMY RAY", "169 EDG ERTON STREET , BAXLEY , GA 31513", > "Appling"); INSERT INTO offenders ( id , name , address, county ) VALUES > (3016, "MATHIS, WILLIE EUGENE", "54 MIDWAY CHURCH RD , BAXLEY , GA 31513", > "Appling"); INSERT INTO offenders ( id , name , address, county ) VALUES > (2203, "PERRY, MATTHEW LAVERNE", "500 CENTRAL STREET , BAXLEY , GA 31513", > "Appling"); INSERT INTO offenders ( id , name , address, county ) VALUES > (1798, "ROBERSON, LARRY B", "528 QUAIL HOLLOW ROAD , BAXLEY , GA 31513", > "Appling"); > MySQL Error: 1064 (You have an error in your SQL syntax near '; INSERT INTO > offenders ( id , name , address, county ) VALUES (5560, "DASHER, A' at line > 1) That isn't one query. That is many queries in one string. I think you need to submit each insert statement seperately. -- Scott Mattocks scott at crisscott.com http://www.crisscott.com http://pear.php.net/user/scottmattocks --------------------------------------------------------------------- Atlanta PHP Talk http://www.atlphp.org/archives/talk http://www.atlphp.org --------------------------------------------------------------------- To unsubscribe, e-mail: talk-unsubscribe at lists.atlphp.org For additional commands, e-mail: talk-help at lists.atlphp.org From ben at atlphp.org Fri Jun 3 11:54:17 2005 From: ben at atlphp.org (Ben Ramsey) Date: Fri, 03 Jun 2005 11:54:17 -0400 Subject: [atlphp-talk] Invalid SQL In-Reply-To: <42A07C0B.8010203@crisscott.com> References: <0MKuxu-1DeEKY171d-0005Wb@mx.perfora.net> <42A07C0B.8010203@crisscott.com> Message-ID: <42A07D29.9020509@atlphp.org> Scott Mattocks wrote: > That isn't one query. That is many queries in one string. I think you > need to submit each insert statement seperately. > MySQL itself can handle executing a group of queries strung together like this, but I'm not sure whether the PHP mysql functions support it. http://www.php.net/mysql_query -- Ben Ramsey Atlanta PHP http://atlphp.org/ --------------------------------------------------------------------- Atlanta PHP Talk http://www.atlphp.org/archives/talk http://www.atlphp.org --------------------------------------------------------------------- To unsubscribe, e-mail: talk-unsubscribe at lists.atlphp.org For additional commands, e-mail: talk-help at lists.atlphp.org From lists at neoncowboy.com Fri Jun 3 11:54:26 2005 From: lists at neoncowboy.com (John Corry) Date: Fri, 3 Jun 2005 11:54:26 -0400 Subject: [atlphp-talk] Invalid SQL In-Reply-To: <42A07C0B.8010203@crisscott.com> Message-ID: > That isn't one query. That is many queries in one string. I > think you need to submit each insert statement seperately. John Corry wrote: >I know I am overlooking something so stupidly simple that it's embarassing to even bring it up.. Thanks. --------------------------------------------------------------------- Atlanta PHP Talk http://www.atlphp.org/archives/talk http://www.atlphp.org --------------------------------------------------------------------- To unsubscribe, e-mail: talk-unsubscribe at lists.atlphp.org For additional commands, e-mail: talk-help at lists.atlphp.org From necrotic at gmail.com Fri Jun 3 11:16:26 2005 From: necrotic at gmail.com (James Logsdon) Date: Fri, 03 Jun 2005 11:16:26 -0400 Subject: [atlphp-talk] Invalid SQL In-Reply-To: <42a07d3b.1087fb85.58eb.ffffa79cSMTPIN_ADDED@mx.gmail.com> References: <42a07d3b.1087fb85.58eb.ffffa79cSMTPIN_ADDED@mx.gmail.com> Message-ID: It works in phpMyAdmin because it splits at the semi-colon to allow multiple queries. On Fri, 03 Jun 2005 11:54:26 -0400, John Corry wrote: > >> That isn't one query. That is many queries in one string. I >> think you need to submit each insert statement seperately. > > John Corry wrote: >> I know I am overlooking something so stupidly simple that it's >> embarassing > to even bring it up.. > > Thanks. > > > > --------------------------------------------------------------------- > Atlanta PHP Talk > http://www.atlphp.org/archives/talk > http://www.atlphp.org > --------------------------------------------------------------------- > To unsubscribe, e-mail: talk-unsubscribe at lists.atlphp.org > For additional commands, e-mail: talk-help at lists.atlphp.org > --------------------------------------------------------------------- Atlanta PHP Talk http://www.atlphp.org/archives/talk http://www.atlphp.org --------------------------------------------------------------------- To unsubscribe, e-mail: talk-unsubscribe at lists.atlphp.org For additional commands, e-mail: talk-help at lists.atlphp.org From ben at atlphp.org Wed Jun 8 07:50:52 2005 From: ben at atlphp.org (Ben Ramsey) Date: Wed, 08 Jun 2005 07:50:52 -0400 Subject: [atlphp-talk] [Fwd: Re: Atlanta PHP Book Reviews] Message-ID: <42A6DB9C.1050902@atlphp.org> Please see the message below. I will serve as the conduit for book requests for review. Send me the O'Reilly book name and ISBN, and I will pass it along to Marsee at O'Reilly. I will wait a week before doing this so that I can have a good list to send her so that we can get all the books at the same time. If you're interested in getting a free copy of an O'Reilly book (so long as you're prepared to review it and post your review to the Atlanta PHP Web site), then now is your chance (of course, we'll continue to do this in the future, as well). Send me your requests (with title and ISBN) off list. We should have the books in time for July's meeting. Thanks! -Ben -------- Original Message -------- Subject: Re: Atlanta PHP Book Reviews Date: Tue, 7 Jun 2005 22:53:02 -0700 From: Marsee Henon To: Ben Ramsey Hi Ben It is easier for me to have one person from each group requesting books. Do you want to put a list together for me? All I need is the book name and ISBN. I don't really have a limit on reviews, but I think it is best to start folks off with one book rather then overwhelm them with a pile. Here are some review tips to pass along too: http://ug.oreilly.com/bookreviews.csp Thanks, Marsee On Jun 3, 2005, at 6:18 AM, Ben Ramsey wrote: > Several of our members are interested in reviewing books for O'Reilly. > These reviews will be posted to our Web site at http://atlphp.org. > > What is the process that we need to follow for this? Should the > requests come through me, or should I have them contact you directly? > One question I've been asked is whether O'Reilly has a limit on how > many reviews one person can do. > > Thanks! --------------------------------------------------------------------- Atlanta PHP Talk http://www.atlphp.org/archives/talk http://www.atlphp.org --------------------------------------------------------------------- To unsubscribe, e-mail: talk-unsubscribe at lists.atlphp.org For additional commands, e-mail: talk-help at lists.atlphp.org From necrotic at gmail.com Wed Jun 8 17:11:35 2005 From: necrotic at gmail.com (James Logsdon) Date: Wed, 08 Jun 2005 21:11:35 -0000 Subject: [atlphp-talk] FreeMeetup.org Message-ID: Did some searching online today for #wordpress and found an alternative to Meetup.com: FreeMeetup.org. It's still in its infancy, but it could be a way to get the word out later on. Just a thought :) James Logsdon --------------------------------------------------------------------- Atlanta PHP Talk http://www.atlphp.org/archives/talk http://www.atlphp.org --------------------------------------------------------------------- To unsubscribe, e-mail: talk-unsubscribe at lists.atlphp.org For additional commands, e-mail: talk-help at lists.atlphp.org From ben at atlphp.org Wed Jun 8 17:43:05 2005 From: ben at atlphp.org (Ben Ramsey) Date: Wed, 08 Jun 2005 17:43:05 -0400 Subject: [atlphp-talk] FreeMeetup.org In-Reply-To: References: Message-ID: <42A76669.7010809@atlphp.org> James Logsdon wrote: > Did some searching online today for #wordpress and found an alternative > to Meetup.com: FreeMeetup.org. It's still in its infancy, but it could > be a way to get the word out later on. > > Just a thought :) I created an Atlanta PHP group there. Let's see how well this thing works or if it gets off the ground. :-) -- Ben Ramsey Atlanta PHP http://atlphp.org/ --------------------------------------------------------------------- Atlanta PHP Talk http://www.atlphp.org/archives/talk http://www.atlphp.org --------------------------------------------------------------------- To unsubscribe, e-mail: talk-unsubscribe at lists.atlphp.org For additional commands, e-mail: talk-help at lists.atlphp.org From lists at neoncowboy.com Fri Jun 10 14:59:09 2005 From: lists at neoncowboy.com (John Corry) Date: Fri, 10 Jun 2005 14:59:09 -0400 Subject: [atlphp-talk] Geocoder and PHP Message-ID: Attn: cross posted to ATLPHP I am working on a project that will require running about 3500 street addresses through a geocoder application to return lat/long pairs. My first idea was to use cURL to post the address to the http://geocoder.us/ site...once for each address, parsing the returned result to get lat/long and store those with the address record in my db. I haven't tried it yet because a quiet but persistent voice is whispering to me that looping through 3500 records and opening a cURL session to another website to screen-scrape the results is somehow not a good idea...for my server or theirs. I don't know enough about all of the issues to be able to articulate why (perhaps you want to enlighten me)...just that it seems like a bad idea. Does anyone know a better way? I can't pay for a commercial service for this but if there's a faster, more efficient means of usign the free services available I'd appreciate hearing about it. Thanks, John Corry --------------------------------------------------------------------- Atlanta PHP Talk http://www.atlphp.org/archives/talk http://www.atlphp.org --------------------------------------------------------------------- To unsubscribe, e-mail: talk-unsubscribe at lists.atlphp.org For additional commands, e-mail: talk-help at lists.atlphp.org From jchum at aismedia.com Fri Jun 10 15:20:10 2005 From: jchum at aismedia.com (Jonathan Chum) Date: Fri, 10 Jun 2005 15:20:10 -0400 Subject: [atlphp-talk] Geocoder and PHP In-Reply-To: <200506101859.j5AIxMWl005878@europa.acs.aismedia.com> Message-ID: <200506101920.j5AJK8Oq007457@europa.acs.aismedia.com> There are zip code databases which provides lat/long values for every zip code. You might want to look into buying one of these databases. The cheapest one I found one company selling it for $29.99 for the June edition. I know it's commercial, but it's not that costly in my opinion. I've used it as a store locator, calculating the distance between 2 zip codes. If their site change's their pattern, you script will also break as a result and I believe is also illegal if they were to find out. Why not run the batch (3,500) only once, store the lat/long pair into a database so you don't have to re-query 3,500 addresses each time? -----Original Message----- From: John Corry [mailto:lists at neoncowboy.com] Sent: Friday, June 10, 2005 2:59 PM To: 'NYPHP Talk' Cc: talk at lists.atlphp.org Subject: [atlphp-talk] Geocoder and PHP Attn: cross posted to ATLPHP I am working on a project that will require running about 3500 street addresses through a geocoder application to return lat/long pairs. My first idea was to use cURL to post the address to the http://geocoder.us/ site...once for each address, parsing the returned result to get lat/long and store those with the address record in my db. I haven't tried it yet because a quiet but persistent voice is whispering to me that looping through 3500 records and opening a cURL session to another website to screen-scrape the results is somehow not a good idea...for my server or theirs. I don't know enough about all of the issues to be able to articulate why (perhaps you want to enlighten me)...just that it seems like a bad idea. Does anyone know a better way? I can't pay for a commercial service for this but if there's a faster, more efficient means of usign the free services available I'd appreciate hearing about it. Thanks, John Corry --------------------------------------------------------------------- Atlanta PHP Talk http://www.atlphp.org/archives/talk http://www.atlphp.org --------------------------------------------------------------------- To unsubscribe, e-mail: talk-unsubscribe at lists.atlphp.org For additional commands, e-mail: talk-help at lists.atlphp.org --------------------------------------------------------------------- Atlanta PHP Talk http://www.atlphp.org/archives/talk http://www.atlphp.org --------------------------------------------------------------------- To unsubscribe, e-mail: talk-unsubscribe at lists.atlphp.org For additional commands, e-mail: talk-help at lists.atlphp.org From lists at neoncowboy.com Fri Jun 10 15:46:40 2005 From: lists at neoncowboy.com (John Corry) Date: Fri, 10 Jun 2005 15:46:40 -0400 Subject: [atlphp-talk] Geocoder and PHP In-Reply-To: <200506101920.j5AJK8Oq007457@europa.acs.aismedia.com> Message-ID: > If their site change's their pattern, you script will also > break as a result and I believe is also illegal if they were > to find out. I'm looking into the legality of it...sent a request to geocoder.us asking how they would like me to do it and if they'd allow me to use their service for my lookup. > Why not run the batch (3,500) only once, store the lat/long > pair into a database so you don't have to re-query 3,500 > addresses each time? Because new addresses are being added constantly. I want to make sure my app is using current data. The geocoder.us app is a webservice that accepts one address at a time as input. I've never worked with webservices in php but I guess it's time to dig into XML-RPC or similar. John --------------------------------------------------------------------- Atlanta PHP Talk http://www.atlphp.org/archives/talk http://www.atlphp.org --------------------------------------------------------------------- To unsubscribe, e-mail: talk-unsubscribe at lists.atlphp.org For additional commands, e-mail: talk-help at lists.atlphp.org From sprocket at f2o.org Fri Jun 10 17:36:45 2005 From: sprocket at f2o.org (Chris Spruck) Date: Fri, 10 Jun 2005 17:36:45 -0400 (EDT) Subject: [atlphp-talk] Geocoder and PHP In-Reply-To: <200506101859.j5AIxU1H032755@mail.f2o.org> References: <200506101859.j5AIxU1H032755@mail.f2o.org> Message-ID: <3395.4.154.91.200.1118439405.squirrel@www.f2o.org> John Corry said: > Attn: cross posted to ATLPHP > > I am working on a project that will require running about 3500 street > addresses through a geocoder application to return lat/long pairs. John, I don't have any real technical insight, but maybe you could meter the speed of grabbing the info? Perhaps do blocks of 35 and spread them out over time? I imagine you could probably script out the interval so you're not having to sit in front of your box, waiting 10 minutes in between launching the next chunk. I took a glance at the geocoder.us site and saw that they mention their API and developer info at http://geocoder.us/help/. That page says "The free service is available for non-commercial uses only" as well as "You should be aware that the free service is actively throttled, and so the commercial service will be faster." so you might be in the clear for just running your 3500 through their system anyway. Hope this helps - let us know how it goes. Chris --------------------------------------------------------------------- Atlanta PHP Talk http://www.atlphp.org/archives/talk http://www.atlphp.org --------------------------------------------------------------------- To unsubscribe, e-mail: talk-unsubscribe at lists.atlphp.org For additional commands, e-mail: talk-help at lists.atlphp.org From ben at atlphp.org Fri Jun 10 17:44:22 2005 From: ben at atlphp.org (Ben Ramsey) Date: Fri, 10 Jun 2005 17:44:22 -0400 Subject: [atlphp-talk] Geocoder and PHP In-Reply-To: <3395.4.154.91.200.1118439405.squirrel@www.f2o.org> References: <200506101859.j5AIxU1H032755@mail.f2o.org> <3395.4.154.91.200.1118439405.squirrel@www.f2o.org> Message-ID: <42AA09B6.5050308@atlphp.org> Chris Spruck wrote: > I took a glance at the geocoder.us site and saw that they mention their > API and developer info at http://geocoder.us/help/. That page says "The > free service is available for non-commercial uses only" as well as "You > should be aware that the free service is actively throttled, and so the > commercial service will be faster." so you might be in the clear for just > running your 3500 through their system anyway. For the easiest solution, use the REST-based approach detailed at the link Chris provided above. If you can use PHP 5, then it's even more simpler since you can use SimpleXML (http://www.php.net/manual/en/ref.simplexml.php). -- Ben Ramsey Atlanta PHP http://atlphp.org/ --------------------------------------------------------------------- Atlanta PHP Talk http://www.atlphp.org/archives/talk http://www.atlphp.org --------------------------------------------------------------------- To unsubscribe, e-mail: talk-unsubscribe at lists.atlphp.org For additional commands, e-mail: talk-help at lists.atlphp.org From ben at atlphp.org Fri Jun 10 21:13:50 2005 From: ben at atlphp.org (Ben Ramsey) Date: Fri, 10 Jun 2005 21:13:50 -0400 Subject: [atlphp-talk] Last call for book reviews (Was Re: [atlphp-talk] [Fwd: Re: Atlanta PHP Book Reviews]) In-Reply-To: <42A6DB9C.1050902@atlphp.org> References: <42A6DB9C.1050902@atlphp.org> Message-ID: <42AA3ACE.6040404@atlphp.org> Well, it's not actually the last call. If this goes over well, we'll probably put in a request for book reviews every other month or so. At any rate, I'll be sending the request to O'Reilly on Monday so we can be sure to have the books available at our July meeting. I've received requests from the following people: Steve Reynolds Chris Spruck Tiffany Brown Callie Carroll James Logsdon If your name isn't listed and you want a free O'Reilly book (that you will also review for our Web site), then send me (off-list) the book title and ISBN by Monday so I can include it in this request. If you don't get it to me by Monday, then don't worry--we can get another request together in a month or two. I've been asked about Apress books, and I don't know whether they have a user group program. I'll check on this and let everyone know. I know that Sams has a user group program that we're members of, but I'm not sure of their policy on requesting books. I know that they have sent me a good number of their books that I had planned on handing out as "door prizes." Perhaps what I should do is list the books I have and give them out on a first-come-first-serve basis to individuals willing to review them (this way, we can add more valuable content to our Web site). Nevertheless, I'll ask Sams their policy on requesting specific titles. -- Ben Ramsey Atlanta PHP http://atlphp.org/ --------------------------------------------------------------------- Atlanta PHP Talk http://www.atlphp.org/archives/talk http://www.atlphp.org --------------------------------------------------------------------- To unsubscribe, e-mail: talk-unsubscribe at lists.atlphp.org For additional commands, e-mail: talk-help at lists.atlphp.org From matt at atlphp.org Sat Jun 11 15:30:29 2005 From: matt at atlphp.org (Matt Kern) Date: Sat, 11 Jun 2005 15:30:29 -0400 Subject: [atlphp-talk] Geocoder and PHP In-Reply-To: <200506101920.j5AJK8Oq007457@europa.acs.aismedia.com> Message-ID: <2FB672A7-6A20-436E-B98C-95A3B281F310@atlphp.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 It really a no-brainer if they supply a free web service. PHP web services are not entirely difficult, just write yourself a nice little consumer for the geocoder service. I wouldn't be all too surprised if there's not already an open source one published somwhere. I have my doubts that they'll help you with a screen scraper utility if they already provide an industry standard method of accessing the data. On Jun 10, 2005, at 3:46 PM, John Corry wrote: > > >> If their site change's their pattern, you script will also >> break as a result and I believe is also illegal if they were >> to find out. >> > > I'm looking into the legality of it...sent a request to geocoder.us > asking > how they would like me to do it and if they'd allow me to use their > service > for my lookup. > > >> Why not run the batch (3,500) only once, store the lat/long >> pair into a database so you don't have to re-query 3,500 >> addresses each time? >> > > Because new addresses are being added constantly. I want to make > sure my app > is using current data. > > The geocoder.us app is a webservice that accepts one address at a > time as > input. I've never worked with webservices in php but I guess it's > time to > dig into XML-RPC or similar. > > John > > > > --------------------------------------------------------------------- > Atlanta PHP Talk > http://www.atlphp.org/archives/talk > http://www.atlphp.org > --------------------------------------------------------------------- > To unsubscribe, e-mail: talk-unsubscribe at lists.atlphp.org > For additional commands, e-mail: talk-help at lists.atlphp.org > > Matt Kern This email has been cryptographically signed. To verify the signature please see http://www.gnupg.org. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (Darwin) iD8DBQFCqzvZveWDDH1muC4RAlrnAJ9YeZan8KXqmAxpNsOpOmarGiYv2gCgjHZL +j9Upw505zPkSOu2yBn6enY= =H22Y -----END PGP SIGNATURE----- --------------------------------------------------------------------- Atlanta PHP Talk http://www.atlphp.org/archives/talk http://www.atlphp.org --------------------------------------------------------------------- To unsubscribe, e-mail: talk-unsubscribe at lists.atlphp.org For additional commands, e-mail: talk-help at lists.atlphp.org From lists at neoncowboy.com Sat Jun 11 19:05:21 2005 From: lists at neoncowboy.com (John Corry) Date: Sat, 11 Jun 2005 19:05:21 -0400 Subject: [atlphp-talk] Geocoder and PHP In-Reply-To: <2FB672A7-6A20-436E-B98C-95A3B281F310@atlphp.org> Message-ID: Yup. Working on it. Will post it in the code session when I'm done. John > -----Original Message----- > From: Matt Kern [mailto:matt at atlphp.org] > Sent: Saturday, June 11, 2005 3:30 PM > To: talk at lists.atlphp.org > Subject: Re: [atlphp-talk] Geocoder and PHP > > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > It really a no-brainer if they supply a free web service. --------------------------------------------------------------------- Atlanta PHP Talk http://www.atlphp.org/archives/talk http://www.atlphp.org --------------------------------------------------------------------- To unsubscribe, e-mail: talk-unsubscribe at lists.atlphp.org For additional commands, e-mail: talk-help at lists.atlphp.org From matt at atlphp.org Sat Jun 11 19:35:45 2005 From: matt at atlphp.org (Matt Kern) Date: Sat, 11 Jun 2005 19:35:45 -0400 Subject: [atlphp-talk] Geocoder and PHP In-Reply-To: <2FB672A7-6A20-436E-B98C-95A3B281F310@atlphp.org> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 I better get the syntax highlighting working then!!! ;-) On Jun 11, 2005, at 7:05 PM, John Corry wrote: > Yup. > > Working on it. > > Will post it in the code session when I'm done. > > John > > This email has been cryptographically signed. To verify the signature please see http://www.gnupg.org. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (Darwin) iD8DBQFCq3VWveWDDH1muC4RAlS1AJ42kYAvQy8Ht0TpC279mjQONzaYYACfYaAt u6oAUzeD9LFFYg+mg7obOjk= =K51g -----END PGP SIGNATURE----- --------------------------------------------------------------------- Atlanta PHP Talk http://www.atlphp.org/archives/talk http://www.atlphp.org --------------------------------------------------------------------- To unsubscribe, e-mail: talk-unsubscribe at lists.atlphp.org For additional commands, e-mail: talk-help at lists.atlphp.org From ben at atlphp.org Sun Jun 12 10:58:19 2005 From: ben at atlphp.org (Ben Ramsey) Date: Sun, 12 Jun 2005 10:58:19 -0400 Subject: [atlphp-talk] Apress user group program Message-ID: <42AC4D8B.309@atlphp.org> I've signed us up for the Apress user group program. As members, we receive 25% off of Apress titles, as well as free review copies of books. To obtain the User Group discount: A.) Place your order with 1-800-SPRINGER or sales at apress.com and make sure to mention the promotion code, "APRESSUG." B.) When placing the order, be sure to mention the NAME (Atlanta PHP) of the user group. For review copies: Send me the title and the ISBN. -- Ben Ramsey Atlanta PHP http://atlphp.org/ --------------------------------------------------------------------- Atlanta PHP Talk http://www.atlphp.org/archives/talk http://www.atlphp.org --------------------------------------------------------------------- To unsubscribe, e-mail: talk-unsubscribe at lists.atlphp.org For additional commands, e-mail: talk-help at lists.atlphp.org