AWStats - analyze server logs. Shared Hosting: AWStats Statistics - Timeweb Help Center Awstats installation
Awstat is a program for building log statistics. Most often it is used to analyze Apache and nginx logs. It is written in perl, and according to the information provided, it is one of the best.
Let's install it from ports:
mkdir /usr/local/etc/awstats/
Now copy the config template:
LogFile="/var/log/httpd/mylog.log"
And go to the path to your log file. It is most convenient to work after the log file is rotated and statistics are built at the end of the day. Therefore, if your log file is immediately compressed with gz, then you need to specify it like this:
LogFile="/var/log/nginx/hilik_access.log.1"
The line that defines what type of log we are working with, for the web server we specify:
She tells awstats that we have the log file in the Apache combined format and matches the nginx main format.
SiteDomain="site"
Enter the main name of your website here. And in the CostAliases, add all the names under which they can still get to your site.
It tells awstat to do address-to-name conversion using the cache file. This makes awstats work faster, once accessing the DNS, it will write the transformation to the file and will subsequently access the DNS server less often.
In the next line, we indicate in which directory we will have the results of awstats, more precisely, intermediate results, visits database, etc. That on the basis of which awstats will generate further displayed statistics.
I usually place it in the same place as the statistics themselves.
DirIcons="http://www.website/awstatsicons/"
On this, in fact, work with the configuration file is completed.
Now we need to make sure that awstatsicons is available.
This can be done in two ways. The first is the simplest, put a symlink or just copy the directory with the awstats icons into the website directory. Or solve this issue with Alias Apache or alias nginx.
For Apache, Alias settings appear in the post install message when installing the awstats port.
They look like this:
| Alias / awstatsclasses "/usr/local/www/awstats/classes/" Alias / awstatscss "/usr/local/www/awstats/css/" Alias / awstatsicons "/usr/local/www/awstats/icons/" |
Alias /awstatsclasses "/usr/local/www/awstats/classes/" Alias /awstatscss "/usr/local/www/awstats/css/" Alias /awstatsicons "/usr/local/www/awstats/icons/"
For nginx, they will look like this:
| location / awstatsclasses/ ( alias / usr/ local/ www/ awstats/ classes/ ; ) location / awstats-icon/ ( alias / usr/ local/ www/ awstats/ icons/ ; ) location / awstatscss/ ( alias / usr/ local /www/awstats/css/ ; ) |
location /awstatsclasses/ ( alias /usr/local/www/awstats/classes/; ) location /awstats-icon/ ( alias /usr/local/www/awstats/icons/; ) location /awstatscss/ ( alias /usr/local /www/awstats/css/; )
These lines must be placed in the settings of the virtual host, the one for which we are building statistics.
Now you need to specify that the index of the directory with statistics will not be index.html or index.php at all, but awstats.site.html, it is under this name that awstats statistics will be generated.
For Apache:
location /awstats/ ( root /usr/local/www/site/; index awstats.site.html; )
Now it remains to make a script that will generate statistics.
It looks like this:
| #!/bin/sh / usr/ local/ www/ awstats/ tools/ awstats_buildstaticpages.pl -lang =ru -config =site -dir =/ usr/ local/ www/ site/ awstats -update |
#!/bin/sh /usr/local/www/awstats/tools/awstats_buildstaticpages..org.ua/awstats-update
And put on cron.
| 15 0 * * * /root/bin/stats.sh |
15 0 * * * /root/bin/stats.sh
This completes the setup.
Small FAQ on AWStats:
Unique visitors
Unique visitors
A unique visitor from whose computer (IP address) at least one request was made to the server during the current report display period (for example, a month). If the visitor has made more than one request, all repeated requests are not counted.
The period for displaying statistics, by default, is one month.
Number of visitors
Number of visits
The number of visits made by all visitors.
In other words, “sessions”, or pages visited by unique visitors who requested any three other pages within an hour. All pages visited by the user during the visit are taken into account, therefore, one should expect many pages per visit and many visits of a unique visitor. A visitor is considered different if there is more than an hour between his requests.
Pages
Pages
Any user view of the page. This includes all pages that are not separately downloadable files (for example, not pictures). Requests for: css js class gif jpg jpeg png bmp ico swf are not taken into account. Viewed HTML pages, CGI, ASP, etc.
requests
Hits
Any page view by the user, including Refresh (Reload) pages. In other words, any call to the web server.
Byte
bandwidth
The total number of bytes sent by the WEB server to the client, including responses such as page not found. Does not count mail or FTP.
Why is the traffic shown by AWStat less than the traffic counted by the provider's billing systems?
Web server visit statistics show the volume of all pages, images and files downloaded from the server. Everything else is not taken into account. The rest includes:
How the TCP/IP protocol stack works. When transmitting data over a network, each piece of data (packet) is supplied with service information, for example: sender and recipient addresses, packet lifetime, data type, checksums and data volume, type of transmission protocols, delivery priority, and so on. Data is transmitted in chunks, after which the sender and recipient exchange confirmations that the data is intact and delivered. In addition, to establish and maintain a connection between the sender and the recipient, the necessary service information that is not related to data is transmitted. If a packet is lost, the data is retransmitted. All this is the service traffic necessary for communication, which must be transmitted for the possibility of data transmission and which must be taken into account by the billing system (since this is full-fledged traffic), but cannot be calculated by the WEB server, since it does not know about it and should not know. Depending on the volume and direction of the transmitted data for this position, discrepancies can reach 1/3 of the AWStat reading.
If several sites are running on the server, then for each site the indications of the amount of data are different, not taken into account by others.
If other services (such as mail, FTP, etc.) are running on the server, their traffic is not taken into account. The volume of this traffic depends on the services themselves and can exceed the volume of site traffic by many times.
When scanning / hacking attempts / attacks, data is transmitted to the server and from it to the hacker, which are not taken into account by AWStat. It is impossible to block such data, but they are full-fledged traffic that will be taken into account by the billing system. The volume of this type of traffic can be either zero or huge, and depends on the interest of hackers in this resource (not necessarily a WEB server, any service running on the server can be an object of interest).
Description:
The report viewing page consists of two frames, the left one contains the report navigation menu, and the central one contains the reports themselves. Each of the AWStats statistical reports is presented in the form of summary values (displayed as a list by default and when called from the frame of the menu of any first-level link) and detailed information.
General
When
Who
Navigation
Referrers
HTTP error statuses
General
The "General" table shows data representing statistics, namely: month, day and time of the first and last visit in a month, data on the number of visits to the site (indicators for displayed and not displayed traffic are given).
Indicators:
Unique visitors
Number of visits (Visits/Visitor)
Pages (Pages/Visit)
Hits (Hits/Visit)
Value (KB/Visit)
When
History for the month
The history for the month (attendance by months) is represented by a graph (histogram) and a table. The history of visiting the resource in each month of the year is shown. In particular, when looking at the month of September, we see a graph and table for each month from January to December, and with values from January to September inclusive. Data for October-December, respectively, is zero.
Indicators:
Unique visitors
Number of visits
Pages
Hits
Value
In the Total column, the total value of each indicator is calculated
Who
Visitors domains/countries (Top 10)
The geography of countries, the number of visitors from different countries is given.
Indicators:
Pages
Hits
Value
Unknown - the visitor could not be recognized. If no other country/region information is displayed other than "Unknown", it means that the GeoIP module has not been connected.
To account for visitors by country/region/city, you need to additional installation and setting up Perl modules and AWStats plugins. AWStats can work with the following plugins:
GeoIPfree (countries), requires Perl module Geo::IPfree
After installing the module, you need to configuration file enable directive LoadPlugin="geoipfree"
GeoIP (countries), you need to install the Geo::IP or Geo::IP::PurePerl Perl module and the GeoIP.dat country database from Maxmind.
After installing the module, you need to include the LoadPlugin="geoip GEOIP_STANDARD /path_to/GeoIP.dat" directive in the configuration file
GeoIP_Region_Maxmind (regions), Perl Geo::IP module and GeoIPRegion.dat region database from Maxmind are required.
After installing the module, you need to include the LoadPlugin="geoip_region_maxmind GEOIP_STANDARD /path_to/GeoIPRegion.dat" directive in the configuration file
GeoIP_City_Maxmind (cities), you need to install the Perl Geo::IP module and the GeoIPCity.dat city database from Maxmind.
After installing the module, you need to include the LoadPlugin=" geoip_city_maxmind GEOIP_STANDARD /path_to/GeoIPCity.dat" directive in the configuration file
In addition, AWStats also supports the following Geo-modules from Maxmind:
GeoIP_ISP_Maxmind (Internet Service Providers)
GeoIP_Org_Maxmind (Organizations)
Hosts (Top 10)
A list of user IP addresses is given. Top 10 - a list of 10 users most frequently visiting the resource is presented.
Indicators:
Pages
Hits
Value
last visit
Robots/Spiders visitors (Top 10)
Duration of visits
The number of users who were continuously on the site in a certain period of time is presented.
The statistics collected the number of visits and the percentage of visits (of the total number of visits) for a period of time:
0 s - 30 s (up to 30 seconds)
30 s - 2 mn (from 30 seconds to 2 minutes)
2 mn - 5 mn (from 2 to 5 minutes)
5 mn - 15 mn (from 5 to 15 minutes)
15 mn - 30 mn (from 15 to 30 minutes)
30 mn - 1 h (from 30 minutes to 1 hour)
1 h+ (more than 1 hour)
Indicators:
Number of visits
Percent
It also shows the total and average number of visits (in seconds).
File type
List of file extensions most frequently downloaded on the site.
Indicators:
Hits and percentage (percentage of total hits)
Value and percentage (percentage of the total number of downloaded MB).
Page URL (Top 10)
It is a list of 10 popular pages of the site.
When clicking on " full list"A list of all pages of the site appears, ranked by the "occurrence" parameter, i.e. by the number of visits through this page, namely this entry point.
It is possible to filter by part of the url (using the "filter" form at the top left) and using the "exclusion filter" form
Clicking on "in" and "out" shows the same list as when clicking on "full list".
It also has the ability to filter data.
When filtering is carried out by specifying a string or using regular expressions.
AWStats supports so-called page aliases, i.e. along with the address displayed in the report, you can also display a predefined page title. To implement this feature, you must include the LoadPlugin="urlalias" directive in the configuration file, then in the folder with the database files (defined by the DirData directive) you must place a text file under the name of the form urlalias.site_address.txt, in each line of which the page address must be written and a tab-delimited title. This module works "on the fly", i.e. does not require regeneration of reports for the changes to take effect.
Operating systems (Top 10)
The number of visitors using different operating systems (OS) is shown (average daily values are displayed according to statistics for the number of days in a month).
Indicators:
Hits
Percentage (of total hits).
Clicking on "full list/versions" shows a complete list of operating systems used, including their versions.
When you click on "unknown", a list of unknown (unidentified) operating systems (rather unidentified OS versions) is displayed. The time of the user's last visit with such an unknown system is also indicated.
Browsers (Top 10)
Shows the number of visitors using different browsers(site viewers). (Daily average values are displayed according to statistics for the number of days in a month).
Indicators:
Graber (search robot or automatic site download systems)
Hits
Percentage (of total hits)
Clicking on "full list/versions" shows a complete list of browser versions in use.
Clicking on "unknown" will display a list called "Unknown browsers (useragent field)". Unrecognized versions of web browsers are listed here.
Referrers
Connecting to a site from
Visits to the site or traffic sources. A table is presented on the ways to access the resource.
Indicators:
Direct Address / Bookmarks
Links from newsgroups
Links from search engines (the top 10 popular search engines are presented; also, when you click on the "full list", a complete list of search engines from which the resource was accessed is displayed). In this case, the top 10 is the same as the full list.
Links from an external page (other websites excluding search engines) are the so-called referring sites where a resource button is installed or links or hyperlinks to a resource are given. Clicking on "full list" displays a complete list of referring sites ( full addresses site pages). Their total number is also indicated (the Total line at the top immediately after the table header).
Unknown origin
Those referrers that came from local addresses (i.e. the document was saved on disk and the link was opened from disk).
Indicators:
Pages/ percentage
Hits/Percentage
Search Keywords (Top 10)
A list of various search phrases is presented. Clicking on "full list" shows the full list of search queries ("Key phrases used by search engines").
Indicators:
Various key phrases
Search (number of requests)
Percent
Search Keywords (Top 10)
Similarly previous paragraph, only search words are presented here. Clicking on "full list" shows the full list of search queries ("Keywords used by search engines").
Indicators:
Various keywords
Search (number of requests), percentage.
Rest
mixed
Article about AWstats web server log analyzer
awstats (A dvanced W eb Stat istics) is log file analysis program web server.
awstats visualizes statistics, including viewing site traffic, unique visitors, pages, clicks, peak hours, search engines, keywords used to find your site, robots, broken links and more.
Installation
- a virtual host configuration file is created on the web server
- restarts the web server to register the link to the reporting site
- A link like this opens in the browser:
- (to improve security) in the /usr/lib/cgi-bin/ folder on the web server, an .htaccess file of the form
Adding the GeoIP module
Connecting an extension GeoIP allows you to increase the information content in the provided report on site traffic and speed up the preparation of reports.
To connect the module GeoIP
- the add-on is installed as described in the article geoip
- Perl interface library is installed
- lines are added to the awstats.conf configuration file
- Setting GeoIP in awstats for a single domain on a FreeBSD server running ISPManager
Description of statistics
Awstats provides information on the following statistics:
- General- Data on the first and last visits for the reporting period, as well as data on the main statistical indicators, divided into displayed and non-displayed traffic. Non-displayed traffic includes traffic generated by robots, viruses or a server response with a special HTTP code.
- Unique visitors- the number of hosts (IP-addresses) that visited the site and viewed at least one page. Host is a single computer or office network, which has a real IP address. If office computers have access to the Internet through a proxy server, then, regardless of the number of computers, only one visit will be counted for the entire office.
- Number of visits- the total number of visitors. A new visitor will be considered a visitor who has not been on the site for more than 60 minutes.
- Pages- the number of pages of the site viewed by each of the visitors. This indicator does not take into account graphics and other files.
- Hits- the number of pages, images and files viewed or downloaded by visitors.
- Value- the amount of data transferred to the visitor
- When: Information on various statistical parameters for the studied period of time.
- History for the month- summary data on statistical indicators for the selected month
- Day of the month- data on statistical indicators divided by days of the month
- Day of the week- average data on statistical indicators divided by days of the week within the selected month.
- Watch- data on statistical indicators divided by hours for the selected day.
- Who: Information about which countries and hosts visitors came from, as well as which robots viewed and indexed the site.
- Countries- data on the countries from which the visitors came. The accuracy of this indicator is not high, since countries are defined by the first level domain.
- Full list- a detailed list of countries, sorted by the number of pages viewed.
- Countries- data on the countries from which the visitors came. The accuracy of this indicator is not high, since countries are defined by the first level domain.
- hosts- data on IP addresses (hosts) from which visitors came
- Full list- a detailed list of known, unknown hosts and unique visitors, sorted by the number of pages viewed
- last visit- list of known, unknown hosts and unique visitors, sorted by date of last visit
- Unresolved IP address- a list of unknown hosts (IP addresses), sorted by the number of pages viewed.
- Robots/Spiders Visitors- data about which search engine robots, when and how many times indexed the site
- Full list- expanded list of visitor robots/spiders, sorted by the number of pages viewed
- last visit- a list of robots/spiders of visitors, sorted by the date of the last visit.
- Navigation: Information about the duration of visitors' visits, the most frequently downloaded file types, the most popular pages of the site, and the operating systems and browsers used by visitors.
- Duration of visits- statistics showing the duration of visits and the number of visitors who stayed on the site for a particular period of time.
- File type- general statistics showing the most frequently downloaded file types.
- Views- statistics showing the most popular pages site
- Full list- expanded list of page addresses, sorted by the number of views
- Entry- a list of pages sorted by "how many times this page viewed first"
- Exit- a list of pages sorted by the feature "how many times this page was last viewed".
- Operating Systems- statistics showing the operating systems used by visitors
- Versions- list of operating systems used by visitors, sorted by operating system version
- Unknown- a list of unidentified operating systems, sorted by the date of the last visit.
- Browsers- statistics showing the browsers used by visitors
- Versions- list of browsers used by visitors, sorted by browser version
- Unknown- list of undefined browsers, sorted by date of last visit.
- Referrers: Information about how visitors got to the site.
- Origin- data about visitors who came from a link from search engines or from a site that links to your
- Referring search engines- data on the number of clicks from search engines, ratings and directories
- Referring Sites- data about the sites that contain links to your site.
- Search- data on key phrases and words by which your site was found in search engines
- Search keywords- list key phrases sorted by number of search attempts
- Search keywords- a list of keywords sorted by the number of search attempts.
- Origin- data about visitors who came from a link from search engines or from a site that links to your
- Rest: Information on statistics that do not fit into any of the above categories.
- mixed- data about plugins used by the visitor's browser.
- HTTP error statuses- data on errors generated by visitor requests
- Pages not found- list of required but not found pages.
Now, as you read these lines, millions of users are searching for Internet solution or an answer to a question. The words they enter into the search engine will determine the search results returned as relevant links that they can then use. If the words that define the search criteria have been entered correctly, the desired Web site will be in the list returned to the user search engine, if the input of words was carried out incorrectly, then no. Do you know what keywords users have previously entered to find your Web site? Do you know which keywords most often serve as pointers to it?
Although the analysis tools web data have been around for a long time, I still never cease to be amazed. The capabilities of the information retrieval and data partitioning mechanisms implemented in modern Web data analysis packages are truly amazing. Want to know how your customers find you? Take a look at the website log analysis and find out which keywords customers use to reach your site. You need to find out what type of operating system is most often found on the computers of site visitors (Windows 2000, Windows 98, Apple Computer Mac OS X or Linux). Web analysis programs can quickly provide such information. All this is good, but what if the company's budget does not allow to purchase software of this type? In such a case, the open source community has an extremely powerful Web data analysis tool that can run under Windows.
The first version of AWStats was developed by Laurent Destel, a computer scientist based in Paris. She appeared in May 2000. Destell needed to receive regular reports on the company's Web data statistics, and because an application was needed that had more capabilities than those programs with open source, which were available at that time in the available sources, he created his own package. Then, in October 2000, Laurent posted source of his Perl program at SourceForge.net, after which this project began to constantly expand and improve. As a result, today the AWStats product has grown to version 6.2 and is on this moment an exceptionally reliable and reliable tool for analyzing log data. AWStats is able to function in the mode command line, but can also be used as an interactive component for Microsoft Internet Information Server (IIS). This product will provide the creators of the Web site with comprehensive information about who visits it.
To demonstrate the procedure for installing the AWStats package, I used Windows 2000 Server with default IIS settings. Thus, the path to the Web site to be analyzed corresponds to C:Inetpubwwwroot, and the log files are located in the standard C:WINNTSystem32LogFiles directory.
Step 1Set up IIS
The only thing that needs to be configured in IIS is the format of the log files. The default log format in IIS is not informative enough for AWStats, so IIS needs to be configured to store log files in the World Wide Web Consortium (W3C) Extended Log Format. This is done in the following way. Start the Microsoft Management Console (MMC) Internet Information Services snap-in, right-click the icon for the Web site whose log settings you want to change, and select Properties. In the Default Web Site Properties dialog box, select W3C Extended Log File Format from the Active log format drop-down list. You then need to click the Properties button, which will open the Extended Logging Properties dialog box. Make sure the following options are selected in this window:
- cs-username
- cs-method
- cs-uri-stem
- cs-uri-query
- sc status
- sc bytes
- cs-version
- cs(User-Agent)
- cs(Referrer)
No other options should be selected. This is due to the fact that AWStat requires a specific data format to work, so any deviation from these requirements may lead to unpredictable results.
To ensure that all log files are in the correct format, you should delete or archive any existing log files that were previously generated by the IIS server. If the IIS server is currently running, it will not be possible to delete the current log file because it is currently in use. In this case, you should simply stop the World Wide Web Publishing service on this system, delete the corresponding file, and then start the service again.
Step 2Install ActivePerl and AWStats
In order to be able to use AWStats, you need to install on this computer the Active Perl package from ActiveState and the AWStats package itself. Active Perl can be downloaded from the ActiveState website at: http://www.activestate.com/products/activeperl. At the time of this writing, there are two versions of this product, ActivePerl 5.8.6 and ActivePerl 5.6.1. In my opinion the 5.6.1 version of ActivePerl works great with the AWStats package and provides the best compatibility with some other software tools in Perl provided by the open source community. Therefore, if you plan to use any other open source Perl applications in your work, I would recommend installing ActivePerl version 5.6.1. However, for those who prefer to have latest versions, I note that AWStats will also work with ActivePerl 5.8.3.
In order to install ActivePerl, you need Windows Installer. The process of installing ActivePerl is simple, the only thing you can choose is the path to the installation directory. I recommend installing ActivePerl in the same directory where the AWStats package is installed by default, namely the C:Program FilesAWStats directory.
After installing ActivePerl, you need to download the main AWStats distribution kit from the SourceForge.net project site (address: ). This kit has executable file, which controls the installation process, so the download procedure is quite simple. Once you've downloaded the AWStats distribution, you just need to run the ".exe" file and accept the default installation directory. After installation is complete, the application will be located in the C:\Program Files\AWStats directory.
Step 3Configure AWStats
During the installation process, AWStats launches its own setup program, which is a Perl application that runs in a command line window. First, as shown in Figure 1, the setup program will ask you for the path to the Web server configuration file. This is due to the fact that this program allows the use of a Web server that stores its parameters in a text file (for example, Apache server). But since this is an IIS server that is not configured in this way, enter none in this box.
| Figure 1: The first window of the AWStats setup program |
After attempting to set up a Web server, this program will guide the user step by step through the process of building a basic template for a Web site. First, as shown in Figure 2, you will be asked if you want to create a new configuration file for AWStats. You should answer Yes. The program will then ask you to enter the name of the Web site to be analyzed. AT this example I chose www.toombspartners.com as the website name Domain name The website of the fictitious organization Toombs Partners. The program uses this name as the name of the corresponding configuration file, so in this case the configuration file will be called awstats.www.toombspartners.com.conf.
At the final stage, the configuration program will report that it will not be able to automatically create scheduled tasks, because the AWStats package is installed on the server with operating system Windows. We will deal with these questions when we perform step 7, so for now you just need to agree with this message, after which the setup program will complete its work. The configuration settings generated in this process are stored in the appropriate configuration file: C:Program filesAWStatswwwrootcgi-binawstats.www.toombspartners.com.conf.
Now we need to do a few advanced settings, so you should open the configuration file with Notepad or another text editor. The first setting to change is the path to the log files. AWStats defaults to the UNIX-specific path /var/log/httpd/mylog.log, so it uses forward slashes (/) instead of backslashes () as delimiters. Find the LogFile parameter in the configuration file, which looks like this:
LogFile="/var/log/httpd/mylog.log"
Replace "/var/log/httpd/mylog.log" with a description of the default path to your system's log files. If IIS was installed with default settings, then "C:/WINNT/System32/LogFiles/W3SVC1/ex%YY-24%MM-24%DD-24.log" should be specified here. This option tells AWStats to run in the C:WINNTSystem32 directory
LogFilesW3SVC1 searches for files that start with ex, end with .log, and have two digits for the year, two digits for the month, and two digits for the day in the middle. This follows the default convention for naming IIS log files. If you are using any other naming convention, or if the log files are stored in a directory with a name other than W3SVC1, then the LogFile parameter must also be changed accordingly.
You then change the value of the LogFormat parameter, which specifies the type of event log for AWStats. The default value here is 1, but for an IIS-based Web site, the correct value is 2.
In addition, you need to edit the SiteDomain and HostAliases parameters, which respectively determine how this site should be referenced from the outside and what are the internal links for this Web site. As the SiteDomain parameter, you need to specify the main domain name of your server or the main name of the site in corporate network. The HostAliases parameter should be edited to contain other valid domain names, addresses, or virtual host names that can be used as internal links to this site.
Step 4Launch AWStats
So, let's assume that all the previous steps have been completed correctly and now you can take the first test run of AWStats. However, before doing this, you should understand how this program works.
When working with the AWstats package, we use two of its main components, namely the analysis program and the reporting program. All mathematical processing of numerical data is performed by the analysis program, while the task of the reporting program is to convert the results of the analysis to the form of intuitive reports in HTML format.
AWStats fully processes information on registered events coming from many Web sites, and enters the results of statistical analysis of this information into a file of its internal database. Accordingly, when we run the AWStats program again, it does not need to reanalyze the previously processed data. Imagine how long it would take AWStats to process the data if it had to reprocess 11 months of log files every time it started in December.
To create the AWStats database and import the IIS log files into it, navigate to the C:ProgramFilesAWStatswwwrootcgi-bin directory and run the command:
awstats.pl -config=WebSiteName
Update -logfile=IISLogName
where WebSiteName is the name of your Web site and IISLogName is the name of the IIS log file to be imported into the database for further analysis. Here this command consists of several lines, in fact, it should be entered on one line. This also applies to other multi-line commands that will be encountered in this article.
Let's say we want to parse the IIS server log files on the Toombs Partners Web site starting in June. Today is June 4th, so we have three completed log files (for June 1, 2, and 3) and one incomplete (June 4 log). To import the file dated June 1, use the command:
config=www.toombspartners.com
logfile="C:/WINNT/system32/
logfiles/w3svc1/ex050601.log"
Then we process the files ex050602.log and ex050603.log in the same way.
The name of the Web site is used by AWStats in the name of the database file. For example, for Toombs Partners, the corresponding database file name would be awstats%MM%YYYY.
www.toombspartners.com.txt, where %MM is a two-digit month and %YYYY is a four-digit year.
When the analysis program starts processing log files, its window will display some statistics on the data being processed. At the end of processing, you will see statistical data, which will indicate the number of new records found by the program (see Figure 3). This number should roughly match the number of lines in the log files being processed. It doesn't matter if a few lines were skipped due to errors. If there are few errors, you can start compiling a report. If, after processing, 0 new records were found, then there is a problem with the configuration file and you need to find out what it is.
Step 5: Create a Web Server to Work with AWS
To make it easier to view reports on the company's Web site, you can build another Web site inside IIS and assign it a directory where the AWStats output will be saved. To create a new directory for AWStats reports, you first need to create a folder named AWStats (ie C:InetpubAWStats) in the Inetpub directory. Then copy the icon folder from the C:Program FilesAWStatswwwroot directory to the C:InetpubAWStats directory. Next, you need to create a new Web site in IIS and configure it to work with this directory, as Figure 4 shows. this server If you already have a site running on port 80, you need to create a unique TCP port, IP address, or new host header for the new Web site.
For the reporting program to work, AWStats creates a special Web page with the default name awstats.WebSiteName.
html, where WebSiteName corresponds to the name of the Web site that we determined in step 3. The name of this page must be set as the name home page by default, which is done through the Documents tab of the AWStats Site Properties dialog box.
Finally, if you need to enter any access restriction settings for the statistics site, this is done through the Directory Security tab of the same AWStats Site Properties dialog box. You can, for example, restrict user access to the site based on IP addresses, or require authentication when accessing the site. In the absence of security restrictions, anyone who connects to this site will be able to view its content.
Step 6. Formation of reports
Now that you've created a Web site in IIS, it's time to practice building reports. Change to the C:\Program Files\AWStats\ools directory and run the following command:
awstats_ buildstaticpages.pl -config=WebSiteName -update -lang=en -dir="C:Inetpubawstats" -awstatsprog="C:/Program Files/AWStats/wwwroot/cgi bin/awstats.pl"
where WebSiteName is the name of your Web site (since this example is Toombs Partners, the site name will be www.toombspartners.com). The reporting component (awstats_buildstaticpages.pl) builds the main statistics report page for the given Web site and places it in the directory specified by the -dir switch.
If all system components are functioning correctly, AWStats will generate reports one by one until the end of the processing cycle is reached. Once this loop is complete, you can launch your Web browser and view the contents of the newly created AWStats Web site. The main page of the AWStats report should be your default page in this case. Fragment example home page statistics is shown on screen 5.
 |
| Figure 5. The main page of the AWStats report |
This page contains statistics high level, as well as links to pages with more detailed information. For example, if you click on the Countries link in Figure 5, which is in the Who section, you can navigate to the Countries (Top 10) table, which is one level down from the main page. By clicking the Full list link in the title bar of this table, you can view a list of countries from which visitors accessed your site. Another way to get to this list is by clicking on the Full list link that follows the Countries link in the Who section.
So, why is there no statistics by country in the report? Don't worry, AWStats behaves this way by default, so let's see how we can fix this.
To determine from which country a visitor is accessing a site, a lookup on the visitor's IP address is used to determine the FQDN (Fully Qualified Domain Name) associated with that address. If such a name is found, it usually ends in something like com, .ca, or .jp. The AWStats program uses this extension to determine which country the visitor is from.
In the default configuration, AWStats does not perform a reverse lookup in the DNS space, because the processing of the corresponding search query for each client IP address that accesses this site, will require a significant amount of time. AWStats is very efficient at doing its internal data processing routines, but in this case it has to send requests to an external DNS server and wait for a response, resulting in slower processing. Consider an example. The three test log files I used to write this article contain about 350,000 addresses. If a reverse DNS lookup is not performed, it takes a few minutes for AWStats to process all this data. If you turn on the reverse search, the process will take more than 6 hours. Therefore, performing a reverse lookup can drastically affect system performance.
However, if you still want AWStats to perform a reverse DNS lookup, you need to find the DNSLookup parameter in the site configuration file and change its value from 2 to 1. After that, AWStats will perform a reverse lookup for each IP address in each newly generated log file when it is parsed. In other words, all existing log files will need to be parsed again. This requires removing existing file AWStats database, and then restart the analysis procedure. In this case, AWStats will send a reverse lookup request for each record, but you need to be prepared for the fact that this will take a long time. Upon completion of this procedure, you should generate reports again, and they will contain data by country for visitors who accessed the site.
Step 7: Set up AWStats to run on a schedule
It is likely that there may be a need to run the log file processing procedure at night, and in such a way that the analysis and reporting procedures are started without human intervention. For these purposes, you can use batch file NightRun.bat shown in listing. This file calls the IIS log file parsing routine, and when it is done, the report builder is invoked, which creates and places them for further processing by AWStats in the IIS directory.
To use the NightRun.bat file, download it from the Windows IT Pro /RE website at http://www.windowsitpro.ru in the download section. Then you need to open the unzipped file with Notepad or another text editor and replace www.toombspartners.com with the name of your Web site. After that, you should launch the Scheduled Tasks application from the Control Panel and create for given file the job is to run it every night at some time after midnight. Why after midnight? The fact is that in step 4 I used the command-line switch -logfile, which was used to set the name of the corresponding log file. If this key is not present, as in the NightRun.bat file, AWStats will automatically select the previous day as the date and use any wildcard that is described by the Logfile= parameter of the configuration file. Therefore, if this batch file is run after midnight on June 4th, AWStats will parse the log files for June 3rd because the date format wildcard specified by the value of the LogFile parameter will be used.
One click - and the web report is ready
How convenient it is to get up-to-date statistics on the Web site every morning! Even for those who are not directly responsible for the content of the company's Web site, such statistics can still be useful, for example, if an employee sells the company's products through the Web site. The information provided by these reports can help you figure out what people are looking for when they visit your Web site, and I think modern customer service should have that kind of information.
This post is likely to be of interest to webmasters hosted on timeweb or those who study this hoster "from afar". I want to talk about the possibilities of Timeweb related to the statistics service awstats.
Working with AWStats statistics logs in the TimeWeb Control Panel
More recently, in the article Calculating traffic on the site, I was faced with the question of how to calculate the traffic eaten by spider robots entering the site server and, in general, how much traffic a web resource consumes. There I deduced some calculation algorithms - a theoretical plan. There is even a formula for an approximate definition of traffic. And today I'll show you how to do it in practice using the log analyzer tool. awstats.
What is AWStats?
A WStats is an analytical report generator for streams on web servers that is freely available to all Internet users. statistics this service provides in human-readable form - in tables and histograms. You can get acquainted with this powerful tool at.
Of course, there are pages on the Web that explain how to install this program on a computer, how to use it, but everything is so complicated there that, frankly, I did not delve into this topic. It seems to me that this is the lot of system administrators.
To colleagues, if you don't mind, take part in the voting at the top of the post. Facebook, Twitter, Vkontakte. Which of them, in your opinion, is cooler in website promotion?
And now I’ll tell you what TimeWeb hosting has to do with it.
But at the present time, I live there with my blog. Therefore, with my question, I decided to look into the TimeWeb Control Panel. As it turned out, statistics collection for AWStats is configured here.
Analyze traffic through TimeWeb statistics logs
In order to receive statistical reports, you first need to enable logs in the Timeweb control panel. Without this action, nothing will simply be collected.
To do this, go to the Management section. LOG files, select the domain or subdomain we need and turn on (tick) access logs, errors and, most importantly, Awstats logs analizer.
This is, in fact, all our actions on TimeWeb.
View AWStats statistics
We will view the reports in the browser. We go to any browser, I, as usual, in Google Chrome, and put the following text into the address bar of the browser: http://domain/awstats , where instead of domain we enter our domain name, for example: site I draw your attention to the name must contain both the second and first levels, with a dot. In this example.ru. I did it: http: // site / awstats W meme Enter and get the login form.
From here, enter the login and password to enter the TimeWeb Control Panel. Click on the Login button.
And we get complete information on the movement on the server of our site. Here's what it looks like in a single browser window.
What can be seen in AWStats?
Well, firstly, just about the traffic. Still, because of him, the fuss flared up. I will say right away, since I recently connected logs on the server, I don’t have reports for the full month yet. I will try to report on the volume of traffic based on the results of April. Here is today's date general table traffic for the month of April (for 3 days).
Please note that non-displayed traffic includes traffic generated by robots.
In general, there are a lot of detailed tables here, see for yourself. Here, for interest, the menu of service sections:
So, the owners of hosting on TimeWEb have the opportunity to track the statistics of visits to their site.
To a webmaster whose hosting is not TimeWeb.
In principle, as far as I understand, AWStats is attached to the APACHE web server, which is connected to all hosts. Therefore, connection to AWStats should be available from any hosting.
Firstly, you can look at the section for connecting log files in the control panel of your hosting, or ask a question to technical support.
Secondly, there is a more general way to view statistics through a browser. Such:
http://domain/awstats/awstats.pl
I won’t say anything, but I tried to log in with my address, everything turned out the same. I also entered the hosting username and password. Of course, log files must be connected.
T now I can track traffic consumption in real, not in calculated volumes. By using Hosting TimeWeb and service awstats. I will keep you informed.
