Using regular expressions to pull out the raw data can be a bit intimidating for the uninitiated and a bit messy as a script can contain a lot of them. At the same time, if you're already familiar with regular expressions, and scrape your project is relatively small, they can be a great solution.

There are some companies (including our own) specific for commercial applications are offered to screen scraping. Applications vary widely, but for medium to large projects, they are often a good solution. Each has its own learning curve, take the time to learn a new application must plan on the ins and outs.

Benefits:

- If you already have a regular expression and be familiar with at least one programming language, it can be a quick solution.
- Regular expression that the content of such small changes will not break them in the "vagueness" to achieve a reasonable amount.
- Data models typically built example, if you are extracting information from websites about cars already extraction engine, model, and rewarding, it easily to existing data structures has been able to identify.

Regular expressions are supported in most modern programming languages. Heck, even is a regular expression engine. It's also good because the various regular expression implementations are not significantly different in their syntax.

Disadvantages:

They do not have much experience with them can be complicated to. Learning regular expressions is not like Perl to Java.

- They are often confusing to analyze.
- The process of data discovery (where data from different web pages you want to get on page crossing) remains to be addressed, and very complex as you can use cookies or similar need.
- To work with relative to such an engine is complex.
- Are expensive to build these types of engines.

Screen scraping software

Benefits:

- The abstract complex things away. Something about regular expressions, HTTP, or cookies without knowing the screen scraping applications can do anything very sophisticated things.
- Setting up the site had to be drastically scaled reduces the amount of time.
- If you run into problems while using a commercial application, screen scraping, chances are that there are support forums and help lines where you can get help.

Disadvantages:

- The learning curve. Each application has its own way to go about things in the screen scraping.
- A possible cost.
- An individual approach.

When the screen scraping applications use this approach to ease of use, price, fitness, and dealing with a wide range of very different scenarios. Chances are however that if you do not mind paying a little bit, you find yourself using one can save a considerable amount of time.

We currently have a project engaged in extracting the newspaper ads work. About the data in the ads as you can get is. However, we had to find the data processing. we decided to use the screen scraper and it's just great to deal with. The basic process that the various pages of the screen scraper site cross dates then inserted into a database.