Web Log Analysis - January 2002

About 3 years ago I wrote an application for my own use. I wanted to know the popularity of the tips I create each month. One measure is the number of times a tip's source code is downloaded - ZIP files. I was also interested in the number of times a particular tip was viewed - HTML files. And about a year ago I added code to show the popularity of 'wallpaper' pictures posted on my site - BMP files. Over the years I have refined the 'application' so I decided to make this application the tip-of-the-month.

Earthlink and Urchin Statistics

My site is hosted by Earthlink. As part of the service statistics are logged in the 'Logs' folder. Earthlink provides software to analyze the logs. The analysis software is called 'Urchin'. More on how I retrieve the statistics below. This application works with the information provided by Urchin. If you are using a different host or different analysis tool you will need to modify the 'input' functions of this application to read your format.

The Web Log Analysis Application

Web statistics are displayed on the following screen:

In this example 'shell.zip' has been downloaded 440 times during the months of September to November 2001.

Left List - The left list contains web statistics files. The files are read from the folder shown above the list - in this example from 'g:\windows\desktop\archive\2001archive\weblog\'. Default is App.Path. To change the folder use the menu File/'Read Input From Folder' - see below. I have included three sample files with the source code. I download the data every few days and store it in a 'monthly' file. The file name should use the following format. The application checks for 'ur' in the first two positions and '.txt' in the last four positions. All files with that file name combination, located in the input folder, are added to the left list. You could save 'weekly' or 'daily' lists.



Urchin_ Required 'ur' in first positions.
yyyy Year
mm Month
.txt Required

Right List - The web statistics are displayed in the right list view. The first column heading contains the name of your local web folder. If you double click on a listed item then the application will attempt to open the file. For example: If you double-clicked on '/v02n05/shell.zip' then the application will attempt to open that file, using WinZip or the associated application, from folder 'g:\windows\desktop\earthlinkweb'. Use file menu item 'Change Local Web File's Folder' to point to your web folder - see below. There is no default local web folder.

Sorting the Right List - You can sort the list by clicking any heading. Click again to reverse the order - ascending to descending and back.

Search - By default, 'zip, bmp, and 'html' are added to the search combo box. In this example we are searching for HTML documents:

Documents ending with 'htm' or 'html' are displayed.

Special Cases - I have coded several 'special cases'. A 'group' function was added to group 'bmp' files which have a common name. And I exclude certain files from the statistics. You may want to examine the code and review those special cases.

Changing the Input Folder

 Use file menu item 'Read Input From Folder', for example:


Changing the Web Folder

Use file menu item 'Change Local Web File's Folder'', for example:


Exporting Log Data Using Earthlink's Urchin

Step 1- Link to Urchin - Launch your browser and link to Urchin, for example, http://www.buygold.net/logs/urchin, then log in. You will see a dialog similar to:

Step2 - Display Pages - Urchin opens with the screen shown below. In this case there were 374 visitors to the site on Wed 12/19. Click the 'pages' button.

From the traffic view click the 'pages' button.

Step 3 - Select Month View - The 'Pages' display opens with the 'weekly' view. Click the 'Month' button.

From the weekly view click the 'month' page view button.

Step 4 - Export Data - From the 'Month' view click the 'Export Data' button. The exported data appears in a separate window.

From the monthly page view click the 'export' button.

Step 5 - Save As - Save the exported data in a file by selecting menu File/Save As...

Urchin data for 2001/12.

Step 6 - Save Dialog - Select a folder into which you will save the Urchin data. See above to select a 'save in' folder.

Saving exported data in file 'Urchin_200112.txt'.

The Demo Program Uses These Modules

This tip-of-the-month uses eight modules which are also tips. Click on any module name to link to the tip's documentation.

Module - Click for More Info Description Used in the Demo To
modConnectToWebSite.bas Connect to a web site Show this info when Help/Info is
selected from the demo.
Browse for folder. To select the input folder
and web site folder.
modAddBackSlash.bas Add a backslash if needed. As necessary.
modLocationLocationLocation.bas Form housekeeping. Set the forms last location and other form
movement functions, for example: Center
modMouseSet.bas Mouse stack. Set and restore the mouse.
modPath.bas Get file name, path or extension. Parse file information.
modShellExecuteUsingFileName.bas Open a file with its associated application. View a file by double-clicking its name.
TBRegistry.Bas Read and write to the registry. Used to save location and
combo box settings.

Download VB 6 Source Code

Click here to download the VB 6 source code [58k]

Previous Tip   Tip Index   Next Tip    First Tip


Updated 2001/12/29