| By Sumantra Roy
Robogen is a software for Windows 95, Windows 98 and
Windows NT based computers which can help you create robots.txt
files without having to worry about the correct syntax
of a robots.txt file. It comes in two editions: the free
Limited edition and the Standard edition which costs $12.95.
Visit the Robogen
Website.
In this article, I'll tell how you can use Robogen to
simplify the process of creating a robots.txt file. This
article assumes that you have read my article on what
a robots.txt file is and how you can manually create it.
If you missed the article, click here to learn
how to create a robots.txt file.
I would recommend that you download the trial version
of the Standard Edition. The Limited edition does not
support as many search engines as the Standard Edition.
Once you have downloaded Robogen and installed it in
your hard drive, start the program. Here are the steps
that you should follow to create a robots.txt file:
Step 1: Click on the Logon FTP Server
button. In the dialog box that appears, type in the
address of the FTP server for your web site, make sure
that the "This server requires me to log in"
option is checked, and then type in the username and
the password which you use to access the server in the
appropriate boxes.
Now, depending on how your web host has set up the server,
the files for your web site may be present in the root
directory of the server itself. Or, it may be present
in a sub-directory of the root directory. In the latter
case, click on the "Use specified root" option
and type the relevant sub-directory in the text box besides
the option. If you don't know in which sub-directory the
files for your web site are present, ask your web host
to give it to you.
Done? Now click on OK.
After a while, Robogen will display the directories and
files which are present in your site in the right pane
of the main Robogen program window. If you find that Robogen
has not displayed the files in your site, it means that
you did not enter the correct sub-directory in which the
files of your web site are present. In this case, ask
you web host to give the correct sub-directory to you.
Step 2: In the left pane of the main
Robogen window, click on the small button present to
the right of the drop-down combo box at the top. In
the menu that appears, select "Popular Search Engines".
Step 3: Now, click on the drop-down
combo box and choose the search engine that you are
targeting (say, AltaVista). In the right pane, select
a page which is NOT meant for AltaVista. Now click on
the "<<<" button. You will find that
the file which you had selected in the right pane is
now present in the left pane. In this way, click on
each of the pages which are not meant for AltaVista
and press the "<<<" button.
Repeat step 3 for each of the search engines that you
are targeting. Just select the search engine that you
are targeting from the drop-down combo box, select the
pages which are not meant for the search engine one by
one from the right pane, and click on the "<<<"
button after clicking on each age. Repeat this process
for all the search engines that you are targeting.
Step 4: Upload the file to your server
using the "Upload Robots.Txt" button. If you
want, you can also use the Save menu item from the File
menu to save a copy of the robots.txt file in your hard
drive.
Article by Sumantra Roy. Sumantra is one of the most
respected search engine positioning specialists on the
Internet. To have Sumantra's company place your site at
the top of the search engines, visit
his site.
Internet
Marketing Home
**
This is a sample of what you'll find in our Affordable,
Members-Only Area **
We offer TONS of great tips, a complete guide to Internet
Marketing, and more!
Membership to our site will allow you
access to numerous reviews on all kinds of
products and services in every category you can imagine.
Join us Today!
If you're not sure yet... be sure to Join
our newsletter for the latest announcements,
additions, and changes to our website. We never accept
"paid ads" and will never sell
your email --as we DETEST Spammers as much -if not more-
than you do! Join Below!
|
