Built For You: Automated Niche Websites With Daily Unique Content!
JLForums.com Exclusive Special Offers

« previous next »

Author Topic: Max number of records?  (Read 640 times)

0 Members and 1 Guest are viewing this topic.

Offline davisTopic starter

  • Trade Count: (0)
  • Newbie
  • *
  • Thank You
  • -Given: 0
  • -Receive: 0
  • Posts: 3
    • View Profile
    • Awards
Max number of records?
« on: June 15, 2009, 12:10:55 AM »
Hi,

I got Web Data Parser to spider what I was looking for (about 350 url's on one site, which works out to 176281 records).  Web Data Parser found all of them, but it gets stuck on "Formatting data, please wait..." and just stays at 1 bar.

What is the maximum number of records it can process at one time?  Should I split this into groups of say 50 url's, and then have it extract all those records, then do the next 50, etc.?

Logged

Offline davisTopic starter

  • Trade Count: (0)
  • Newbie
  • *
  • Thank You
  • -Given: 0
  • -Receive: 0
  • Posts: 3
    • View Profile
    • Awards
Re: Max number of records?
« Reply #1 on: June 15, 2009, 12:37:43 AM »
Ahhh...50's not working either, the program just stops responding...
Logged

Offline davisTopic starter

  • Trade Count: (0)
  • Newbie
  • *
  • Thank You
  • -Given: 0
  • -Receive: 0
  • Posts: 3
    • View Profile
    • Awards
Re: Max number of records?
« Reply #2 on: June 16, 2009, 06:18:09 PM »
Finally got this working by making sure it never did more than 15,000 url's per extraction.

In the future, it'd be nice if there were an option for setting the number of threads or if there were some way to make it process more records per extraction.
Logged

JL Forums

Re: Max number of records?
« Reply #2 on: June 16, 2009, 06:18:09 PM »
« previous next »