Installing Search_Engine Script on Win95/98 Servers

Unpublished work. Permission granted to use and modify this library so long as the copyright above is maintained modifications are documented, and credit is given for any use of the library.

I've written this installation document after realizing the need for documentation on my first installed script from the author above. Let me begin by noting that Steven Brenner's work is appreciated in writing such a fine script that was easily integrated with the Windows 98 environment server.

This script currently runs well on my site as well as others in both unix and win32 platforms. Several people have emailed me with questions about installation on unix servers. The scope of this documentation is to assist on installation with windows based webservers, namely Win32 platforms, specifically.

Addendum: (June 29,2001): I have added support for .ASP file extensions as well as .htm, .html files into the search_engine.pl file. See lines 294-298 of search_engine.pl which are the changes I have made to this code to support the .asp extension:

if (($filename=~/asp.?/i) ||
   ($filename=~/htm.?/i) ||
     (!($filename =~ /^\.\.?$/) &&
      -d "$directory/$filename")) {
last;
}

The changes are reflected in the file that can be downloaded here of the script.

System Requirements:

Win32 based system (Windows9x/NT/2000)
A web server installed - Tested compliant with OmniHTTPd from Onicromn Technologies.
Perl v5.005 (earlier versions haven not been tested)
Keyword_search_4.0.tar file, available from www.extropia.com
Internet service provider is optional for testing scripts, but not required.

Extracting the files

Using Winzip, extract the archive to a folder for your web server, allowing the archive to use foldernames embedded in the archive.
The main portions of files that will require editing are the following:

Search-engine.cgi - This is the actual script which performs the calls, and usually requires only minimal editing.
Search-define.cgi - The definition file to set directories, and to configure the script to 'not look' at certain files/directories.

Open the search-engine.cgi using your favorite text editor.The first line of the script is the only one you should edit in the search-engine.cgi file:

# !C:\Perl\5.005\bin\MSWin32-x86-object\perl.exe -T

CAUTION The -T option may not work with your version of Perl. To test if the -T option works, run the script from a command line. If you get an error from perl, simply remove the -T option. Or you may want to replace the -T with the -w option to catch errors on the screen.

This is the literal path statement for the script to find the perl.exe file. It may not be required, but I always place this line in every script I install, and it has not failed me yet. Do it. Do it first, and do it for every script. Do it.

Save the file, and close it. You are now done with this file. Do not make any changes to this file at this time.

Open search-define.cgi using your favorite text editor.

This file Defines Variables for the search_engine.cgi script, and must be edited to meet your needs. The script will not function properly right out of the box. Please do not contact me with complaints that the script is not working if you've not even gotten this far in the installation yet!

Locate the following line, within the top paragraphs of the script:

$root_web_path = "/your root/rootfolder/";

You must use the "/" instead of the "\" for the real path to where your web server is storing it's root directory. This is the literal path to the servers' root directory. This is where the search engine begins searching for HTML, HTM, and all other files. All sub-folders beneath your root are then searched, in order of the tree, unless specified later in the @unwanted_files array.

What is the server root???

The server root is the first folder that contains the start of your file for your web site. It contains the index.shtml, htm, html (or whatever) file that calls your first page on the server when visitors come to your site.

Example: My server root directory is C:\httpd\HtDocs, which is the actual directory path to the folder that contains the first file of your website.
The line would then read:

$root_web_path = "c:/httpd/htdocs";

Remember, Windows 95/98 servers are not the same as Unix servers, but Perl doesn't care about the OS, and reads all directory trees with the "/" not the "\".

$server_url = "http://hawkwynd.com";

This line points to the exact URL of your site. Change this line to reflect your own URL to your top level page.

Unwanted Files to be excluded from the search engine (search-define.cgi)

Some confusion is generated here, when trying to configure the unwanted folders and files from the search. The @unwanted_files array stores all the directories, and individual filenames that the script is to 'ignore' when searching files through the directory tree. Some examples and explanations:

@unwanted_files = ("index.html",
"\images\",
"\keywords\search-engine\index(.*)html");

This line above tells the script to ignore all files "index.html" as well as the \images\ directory, and ignore any file starting with or containing index contained in the \keywords\search-engine subdirectory.

@unwanted_files = ("Error(.*)html","error(.*)html");

Here we'll set the script to NOT search these files. To add more files, follow the scheme above, and use the literal path to the files location. You can set entire directories to be excluded:

@unwanted_files = ("/HtDocs/","/cgi-bin","/keyword/search.html");

The above line tells the script to skip the /HtDocs folder as well as the /cgi-bin folder, and skip the search.html file in the /keyword folder. Setting the proper unwanted files will speed up your engine, and provide greater security from unwanted information being viewed.

The remainder of the script does not require editing to perform on your server. There are some HTML codes further down that you may wish to modify for cosmetic reasons, but the remainder of the cgi code should be left intact.

CALLING THE SCRIPT FROM YOUR PAGE

To activate the script, use the following syntax: ( or create your own )

<form action="/keyword/search_engine.cgi" method="GET">
<input type="text" size="12" maxlength="50" name="keywords">
<input type="image" name="SUBMIT" src="/icons/search_off.gif">

USING META TAGS TO OPTIMIZE SEARCHES

At this point, you are ready to make the needed changes to your META tags in your pages.
For each page that you wish to be searched, include the following in the <head> </head> section.

Example for my weather page:

<HEAD>
<META NAME="DESCRIPTION" CONTENT="Search any major U.S. city for the latest weather forecast">
<META NAME="KEYWORDS" CONTENT="weather, search, forecast, snow, rain, sleet, hail, sun, clouds, wind">
</HEAD>

Continue using these tags, which the search script will locate without having to scan the entire page of code for the words being searched.

For more help with script installations on a Windows 95/98 based server, write hawkwynd.

Tips for troubleshooting

Before sending any correspondence about this script, please know that I am not the original author of this script. The original author can be reached at http://www.extropia.com/. Please consider this before writing me with your problems/questions regarding this script.

Now that I've explained that, here's some things to think about before you go pulling your hair out because the script is not working for you.

1. Use the -w option in the perl bang path. This often will reveal typos, or other warnings to alert you to why your script is not working properly.
2. If you really want to get anal, use the following line in the search_engine.cgi script:

use strict;

3. If the script does not report errors, however doesn't return matches to your queries, check the @unwanted_files array configuration in the search_define.cgi. Cehck to make sure your specifications are properly entered.

4. Still no luck? Here's how to best get the answers you need quickly:

When you write, please include the following information:

Operating system of the server
Web server software being used
Error messages, if any being reported
Snippet of code or entire file(s) (both files preferred)