Hello,
This is one of those questions that seems as though it must have an obvious answer, but alas...I am looking for a relatively long term answer, if such a thing can possibly exist in the software world.
So, here is the question: how do you write/deploy a program on Godaddy/Linux that reads web pages? I have a partial answer for old-world Visual Basic. There is an object that comes with Visual Basic, called an INET object, that reads pretty much any URL. If the URL goes to a web page, you can read the URL into a string, then search the string for whatever you are looking for. If it is an image, you can save it as a binary file. That is old-world Visual Basic.
I am thinking there has to be an elegant way put a program on Godaddy/Linux that will read a webpage, then save the result in MySQL. I want to run this on my hosting account at Godaddy, so the saved results can be used as content on webpages. My hosting is Linux on Godaddy, not Windows.
TEST CASE: Make a program that logs into Twitter as me, reads my latest post, and saves it to MySQL in my Godaddy hosting account.
CONSTRAINT
I am moving off Microsoft Windows as my programmer's box, onto Mac OS X., so all the Microsoft .NET stuff is off the table. .NET is hilarious.
BACKGROUND - WHY 300 POINTS FOR THIS QUESTION
I am not looking for the code to get this done, but rather, I am trying to decide what kind of programmng to use for this web-reading capability. Should I be thinking PERL? JAVA? LISP? C? CANARY? PYTHON? RUBY? ON RAILS? I am just using the Twitter thing as a pilot/prototype.
One of my Internet theorie is that programs to read web pages, find data, save the data, and then make the data available as content will be all the rage, if they aren't already. I know a guy at a Silicon Valley venture capitaled startup... they have a JAVA program that reads the web, looking for certain products, which they in turn offer for sale to others. My friend spends his time tweaking the program. How they figure they can make money on that is beyond me, but on another level, I can see that if you can run programs that replace the work of people gathering data via the internet for mass use, it could make a lot of things economical that otherwise would not be. One of my questions is: WHY JAVA? Maybe it's because the guy who started the company is Mr. JAVA, or maybe, I am wondering, is there an INET sort of thing in JAVA that enables reading of web pages, thus saving 1,400 person-years of programming. I really do not want to invent wheels that already are available, no margin in that.
What programming/software architecture would you recommend?
All awesome responses awarded points. Awesome just means serious thought. Sometimes, one word will prove to be the answer after a long thread, and thus will be MAXIMUS AWESOMUS. That happened when I was looking for a wizard to webify SQL Server data, someone typed ASP-Runner, and that was the answer I ended up running with.
BONUS POINTS OPPORTUNITY
Intelligent comments regarding the aforementioned INET object in World Microsoft will be rewarded with points proportional to the intelligence shown. Humor scores double.
Start Free Trial
