*feeders for Magnesia*

These scripts have specific purposes:

"builder" created a new database from scratch. First torrent is the oldest available, 15. Last is latest available (got through RSS). The id is incremented until first hits last, for each torrent info and stats are extracted.
"builder_nexttorrents" extends an existing database with newer torrents. First is got via "id.txt" or user prompt. Last is latest available (got through RSS). 
"builder_refresh" provides an existing database with fresh stats (seeders, leechers, downloadcount). If available it parses each "id.txt" line, else user prompt. It does delete previous stat* files.

Execution time depends on many factors, but expect "builder" to run for a couple of days ! If execution stops at some point, you don't have to delete the files and restart from zero: just use "builder_nexttorrents" instead, after making sure the database files are sane. Please note that "builder_refresh" can't be resumed: if the execution stops, it needs to start from zero next time. You may want to clone the script and modify it to only refresh the most recent torrents, only the ones from 2012 for example, since older torrent stats don't change much.

IMPORTANT NOTICE

To execute these scripts, you'll need:
- GNU/Linux system with bash interpreter
- read/write/execution permissions in the current directory
- wget with internet access
- common tools (cat, awk, sed, head, tail, timeout)
- "Torrent.php" (torrent-rw, included)
- "urlencode.sed" (included)
- a command line interpreter for PHP  (on Debian/Ubuntu:  "sudo apt-get install php5-cli" )  <-- don't forget this, since it's needed to extract torrent name, size and btih


What you get:
A data/ subdirectory containing the data files, like "info104.php" and "stat188.php". Their number and names depend on the id, since each file can contain up to 1000 torrent informations. At the time of writing, a complete database would consist of 650 files, for a total size of 105 MB. Pretty compact for this amount of data, actually.

What to do with it:
You can bulk upload the files to your Magnesia mirror, or zip them. Zip the whole data/ directory (as "data.zip", including the directory itself)), send the zip to your Magnesia mirror and execute the "zipper" script to extract the data out of it (already existing files are replaced, don't worry with database refreshing and update). A zipped archive's size is around 22 MB, at time of writing.

What you can also do with it:
PRIOR TO EXTRACTION on your hosting, copy it and rename to "export.zip": this makes the file available from the "export" link on Magnesia. For security reasons "zipper.php" does delete "data.zip" after extraction, so watch out or you'll have to upload the zip twice !


More info can be found on the wiki:
http://wiki.suumitsu.eu/doku.php?id=php:magnetik_nyaatorrents
