|
חבר וותיק
|
דטאבייס של 140 אלף קישורי הורדות (ראפידשר מגה אפלואד וכו')
פרסמתי את זה באנגלית אז אני פשוט יצטט
Hi,
I have made a robot 2 weeks ago which fetchs and index'es rapidshare, megaupload and other free-upload websites download links into database table.
It runs automaticly each 5 hours and collect the links from about 20-30 websites.
In 2 weeks I have this:
100,000 links, indexed in mysql table.
The table takes 20MB of space
The table scheme is like this:
קוד:
-- --------------------------------------------------------
--
-- Table structure for table `search`
--
CREATE TABLE IF NOT EXISTS `search` (
`id` int(11) NOT NULL auto_increment,
`time` int(10) default NULL, // the insertion time in unixtimestamp format
`unique` varchar(255) collate utf8_unicode_ci default NULL, // the website unique file id
`site` varchar(50) collate utf8_unicode_ci default NULL, // the website unique name
`url` varchar(255) collate utf8_unicode_ci default NULL, // the file full url
`filename` varchar(255) collate utf8_unicode_ci default NULL, // the filename, might be blank
`snippet` varchar(255) collate utf8_unicode_ci default NULL, // the context, usually title, snippet from google search, description, etc
`tags` varchar(255) collate utf8_unicode_ci default NULL, // tags like "porno", "gays", "hot chicks" etc, usually blank
`password` varchar(255) collate utf8_unicode_ci default NULL, // if the robot found out there is password, this is where it saves it
PRIMARY KEY (`id`)
) ENGINE=MyISAM DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci AUTO_INCREMENT=51048 ;
INSERT queries examples from the latest links:
קוד:
INSERT INTO `search` VALUES (null, 1225466050, 'WORJGWTK', 'megaupload', 'http://www.megaupload.com/?d=WORJGWTK', '', 'Supernatural S04E06 HDTV XviD-NoTV', '', 'areg'),(null, 1225466050, 'WORJGWTK', 'megaupload', 'http://www.megaupload.com/?d=WORJGWTK', '', 'Supernatural S04E06 HDTV XviD-NoTV', '', 'areg'),(null, 1225466050, 'X8JGOPLW', 'megaupload', 'http://www.megaupload.com/?d=X8JGOPLW', '', 'Supernatural S04E06 HDTV XviD-NoTV', '', 'areg'),(null, 1225466050, 'X8JGOPLW', 'megaupload', 'http://www.megaupload.com/?d=X8JGOPLW', '', 'Supernatural S04E06 HDTV XviD-NoTV', '', 'areg'),(null, 1225466050, 'W6SPVVBA', 'megaupload', 'http://www.megaupload.com/?d=W6SPVVBA', '', 'Supernatural S04E06 HDTV XviD-NoTV', '', 'areg'),(null, 1225466050, 'W6SPVVBA', 'megaupload', 'http://www.megaupload.com/?d=W6SPVVBA', '', 'Supernatural S04E06 HDTV XviD-NoTV', '', 'areg'),(null, 1225466050, 'P8NBWZ3X', 'megaupload', 'http://www.megaupload.com/?d=P8NBWZ3X', '', 'Supernatural S04E06 HDTV XviD-NoTV', '', 'areg'),(null, 1225466050, 'P8NBWZ3X', 'megaupload', 'http://www.megaupload.com/?d=P8NBWZ3X', '', 'Supernatural S04E06 HDTV XviD-NoTV', '', 'areg');
Code:
קוד:
INSERT INTO `search` VALUES (null, 1225466050, '156975650', 'rapidshare', 'http://rapidshare.com/files/156975650/sprntl406-notv.part1.rar', 'sprntl406-notv.part1.rar', 'Supernatural S04E06 HDTV XviD-NoTV', '', 'areg'),(null, 1225466050, '156975660', 'rapidshare', 'http://rapidshare.com/files/156975660/sprntl406-notv.part2.rar', 'sprntl406-notv.part2.rar', 'Supernatural S04E06 HDTV XviD-NoTV', '', 'areg'),(null, 1225466051, '156975651', 'rapidshare', 'http://rapidshare.com/files/156975651/sprntl406-notv.part3.rar', 'sprntl406-notv.part3.rar', 'Supernatural S04E06 HDTV XviD-NoTV', '', 'areg'),(null, 1225466051, '156975597', 'rapidshare', 'http://rapidshare.com/files/156975597/sprntl406-notv.part4.rar', 'sprntl406-notv.part4.rar', 'Supernatural S04E06 HDTV XviD-NoTV', '', 'areg');
(the auto increment is kinda old because I fetched this information from an old txt file)
FAQ
Q: Are there any doubled links?
A: There are no double / duplicated links since the robot checks the database for duplicates before adding the links.
Q: Are there any removed (aka no-working) links?
A: There database has NOT been checked for not working links, but the links are from the latest 14 days, so personally, I don't think there are any not-working or removed links.
I even don't know how to quote this thing, so you can just come up with price tags and I may proceed with the sell (;
__________________
|