Microsoft Bob

Just a short, simple blog for Bob to share some tips and tricks.

Be sure to check out my non-technical blog at www.bobsbasement.net.

Month List

Automating the Creation of FTP User Isolation Folders

A customer asked me a question a little while ago that provided me the opportunity to recycle some code that I had written many years ago. In so doing, I also made a bunch of updates to the code to make it considerably more useful, and I thought that it would make a great blog.

Here's the scenario: a customer had hundreds of user accounts created, and he wanted to use the FTP service's User Isolation features to restrict each user to a specific folder on his FTP site. Since it would take a long time to manually create a folder for each user account, the customer wanted to know if there was a way to automate the process. As it turns out, I had posted a very simple script in the IIS.net forums several years ago that did something like what he wanted; and that script was based off an earlier script that I had written for someone else back in the IIS 6.0 days.

One quick reminder - FTP User Isolation uses a specific set of folders for user accounts, which are listed in the table below.

User Account TypesHome Directory Syntax
Anonymous users %FtpRoot%\LocalUser\Public
Local Windows user accounts

(Requires Basic authentication.)

%FtpRoot%\LocalUser\%UserName%
Windows domain accounts

(Requires Basic authentication.)

%FtpRoot%\%UserDomain%\%UserName%

Note: %FtpRoot% is the root directory for your FTP site: for example, C:\Inetpub\Ftproot.

That being said, I'm a big believer in recycling code, so I found the last version of that script that I gave to someone and I made a bunch of changes to it so it would be more useful for the customer. What that in mind, here's the resulting script, and I'll explain a little more about what it does after the code sample.

Option Explicit

' Define the root path for the user isolation folders.
' This should be the root directory for your FTP site.
Dim strRootPath : strRootPath = "C:\Inetpub\wwwroot\"

' Define the name of the domain or the computer to use.
' Leave this blank for the local computer.
Dim strComputerOrDomain : strComputerOrDomain = ""

' Define the remaining script variables.
Dim objFSO, objCollection, objUser, objNetwork, strContainerName

' Create a network object; used to query the computer name.
Set objNetwork = WScript.CreateObject("WScript.Network")

' Create a file system object; used to creat folders.
Set objFSO = CreateObject("Scripting.FileSystemObject")

' Test if the computer name is null.
If Len(strComputerOrDomain)=0 Or strComputerOrDomain="." Then
  ' If so, define the local computer name as the account repository.
  strComputerOrDomain = objNetwork.ComputerName
End If

' Verify that the root path exists.
If objFSO.FolderExists(strRootPath) Then

  ' Test if the script is using local users.
  If StrComp(strComputerOrDomain,objNetwork.ComputerName,vbTextCompare)=0 Then
    ' If so, define the local users container path.
    strContainerName = "LocalUser"
    ' And define the users collection as local.
    Set objCollection = GetObject("WinNT://.")
  Else
    ' Otherwise, use the source name as the path.
    strContainerName = strComputerOrDomain
    ' And define the users collection as remote.
    Set objCollection = GetObject("WinNT://" & strComputerOrDomain & "")
  End If

  ' Append trailing backslash if necessary.
  If Right(strRootPath,1)<>"\" Then strRootPath = strRootPath & "\"
  ' Define the adjusted root path for the container folder.
  strRootPath = strRootPath & strContainerName & "\"

  ' Test if the container folder already exists.
  If objFSO.FolderExists(strRootPath)=False Then
    ' Create the container folder if necessary.
    objFSO.CreateFolder(strRootPath)
  End If

  ' Specify the collection filter for user objects only.
  objCollection.Filter = Array("user")

  ' Loop through the users collection.
  For Each objUser In objCollection
    ' Test if the user's account is enabled.
    If objUser.AccountDisabled = False Then
      ' Test if the user's folder already exists.
      If objFSO.FolderExists(strRootPath & "\" & objUser.Name)=False Then
        ' Create the user's folder if necessary.
           objFSO.CreateFolder(strRootPath & "\" & objUser.Name)
         End If
       End If
     Next

End If

I documented this script in great detail, so it should be self-explanatory for the most part. But just to be on the safe side, here's an explanation of what this script is doing when you run it on your FTP server:

  • Defines two user-updatable variables:
    • strRootPath - which specifies the physical path to the root of your FTP site.
    • strComputerOrDomain - which specifies the computer name or the domain name where your user accounts are located. (Note: You can leave this blank if you are using local user accounts on your FTP server.)
  • Creates a few helper objects and determines the local computer name if necessary.
  • Checks to see if the physical path to the root of your FTP site actually exists before continuing.
  • Creates a connection to the user account store (local or domain).
  • Determines the container folder name that be the parent directory of user account folders, and creates it if necessary. (See my earlier note about the folder names.)
  • Defines a filter for user objects in the specifies account repository. (This removes computer accounts and such from the operation.)
  • Loops through the collection of user accounts, checks each account to see if it is enabled, and creates a folder for each user account if it does not already exist.

That's all for now. ;-]

Note: This blog was originally posted at http://blogs.msdn.com/robert_mcmurray/
Posted: Mar 28 2013, 16:29 by Bob | Comments (0)
  • Currently 0/5 Stars.
  • 1
  • 2
  • 3
  • 4
  • 5
Filed under: IIS | Scripting | FTP
Social Bookmarks: E-mail | Kick it! | DZone it! | del.icio.us

Advanced Log Parser Part 7 - Creating a Generic Input Format Plug-In

In Part 6 of this series, I showed how to create a very basic COM-based input format provider for Log Parser. I wrote that blog post as a follow-up to an earlier blog post where I had written a more complex COM-based input format provider for Log Parser that worked with FTP RSCA events. My original blog post had resulted in several requests for me to write some easier examples about how to get started writing COM-based input format providers for Log Parser, and those appeals led me to write my last blog post:

Advanced Log Parser Part 6 - Creating a Simple Custom Input Format Plug-In

The example in that blog post simply returns static data, which was the easiest example that I could demonstrate.

For this follow-up blog post, I will illustrate how to create a simple COM-based input format plug-in for Log Parser that you can use as a generic provider for consuming data in text-based log files. Please bear in mind that this is just an example to help developers get started writing their own COM-based input format providers; you might be able to accomplish some of what I will demonstrate in this blog post by using the built-in Log Parser functionality. That being said, this still seems like the best example to help developers get started because consuming data in text-based log files was the most-often-requested example that I received.

In Review: Creating COM-based plug-ins for Log Parser

In my earlier blog posts, I mentioned that a COM plug-in has to support several public methods. You can look at those blog posts when you get the chance, but it is a worthwhile endeavor for me to copy the following information from those blog posts since it is essential to understanding how the code sample in this blog post is supposed to work.

Method NameDescription
OpenInput Opens your data source and sets up any initial environment settings.
GetFieldCount Returns the number of fields that your plug-in will provide.
GetFieldName Returns the name of a specified field.
GetFieldType Returns the datatype of a specified field.
GetValue Returns the value of a specified field.
ReadRecord Reads the next record from your data source.
CloseInput Closes your data source and cleans up any environment settings.

Once you have created and registered a COM-based input format plug-in, you call it from Log Parser by using something like the following syntax:

logparser.exe "SELECT * FROM FOO" -i:COM -iProgID:BAR

In the preceding example, FOO is a data source that makes sense to your plug-in, and BAR is the COM class name for your plug-in.

Creating a Generic COM plug-in for Log Parser

As I have done in my previous two blog posts about creating COM-based input format plug-ins, I'm going to demonstrate how to create a COM component by using a scriptlet since no compilation is required. This generic plug-in will parse any text-based log files where records are delimited by CRLF sequences and fields/columns are delimited by a separator that is defined as a constant in the code sample.

To create the sample COM plug-in, copy the following code into a text file, and save that file as "Generic.LogParser.Scriptlet.sct" to your computer. (Note: The *.SCT file extension tells Windows that this is a scriptlet file.)

<SCRIPTLET>
  <registration
    Description="Simple Log Parser Scriptlet"
    Progid="Generic.LogParser.Scriptlet"
    Classid="{4e616d65-6f6e-6d65-6973-526f62657274}"
    Version="1.00"
    Remotable="False" />
  <comment>
  EXAMPLE: logparser "SELECT * FROM 'C:\foo\bar.log'" -i:COM -iProgID:Generic.LogParser.Scriptlet
  </comment>
  <implements id="Automation" type="Automation">
    <method name="OpenInput">
      <parameter name="strFileName"/>
    </method>
    <method name="GetFieldCount" />
    <method name="GetFieldName">
      <parameter name="intFieldIndex"/>
    </method>
    <method name="GetFieldType">
      <parameter name="intFieldIndex"/>
    </method>
    <method name="ReadRecord" />
    <method name="GetValue">
      <parameter name="intFieldIndex"/>
    </method>
    <method name="CloseInput">
      <parameter name="blnAbort"/>
    </method>
  </implements>
  <SCRIPT LANGUAGE="VBScript">

Option Explicit

' Define the column separator in the log file.
Const strSeparator = "|"

' Define whether the first row contains column names.
Const blnHeaderRow = True

' Define the field type constants.
Const TYPE_INTEGER   = 1
Const TYPE_REAL      = 2
Const TYPE_STRING    = 3
Const TYPE_TIMESTAMP = 4
Const TYPE_NULL      = 5

' Declare variables.
Dim objFSO, objFile, blnFileOpen
Dim arrFieldNames, arrFieldTypes
Dim arrCurrentRecord

' Indicate that no file has been opened.
blnFileOpen = False

' --------------------------------------------------------------------------------
' Open the input session.
' --------------------------------------------------------------------------------

Public Function OpenInput(strFileName)
    Dim tmpCount
    ' Test for a file name.
    If Len(strFileName)=0 Then
        ' Return a status that the parameter is incorrect.
        OpenInput = 87
        blnFileOpen = False
    Else
        ' Test for single-quotes.
        If Left(strFileName,1)="'" And Right(strFileName,1)="'" Then
            ' Strip the single-quotes from the file name.
            strFileName = Mid(strFileName,2,Len(strFileName)-2)
        End If
        ' Open the file system object.
        Set objFSO = CreateObject("Scripting.Filesystemobject")
        ' Verify that the specified file exists.
        If objFSO.FileExists(strFileName) Then
            ' Open the specified file.
            Set objFile = objFSO.OpenTextFile(strFileName,1,False)
            ' Set a flag to indicate that the specified file is open.
            blnFileOpen = true
            ' Retrieve an initial record.
            Call ReadRecord()
            ' Redimension the array of field names.
            ReDim arrFieldNames(UBound(arrCurrentRecord))
            ' Loop through the record fields.
            For tmpCount = 0 To (UBound(arrFieldNames))
                ' Test for a header row.
                If blnHeaderRow = True Then
                    arrFieldNames(tmpCount) = arrCurrentRecord(tmpCount)
                Else
                    arrFieldNames(tmpCount) = "Field" & (tmpCount+1)
                End If
            Next
            ' Test for a header row.
            If blnHeaderRow = True Then
                ' Retrieve a second record.
                Call ReadRecord()
            End If
            ' Redimension the array of field types.
            ReDim arrFieldTypes(UBound(arrCurrentRecord))
            ' Loop through the record fields.
            For tmpCount = 0 To (UBound(arrFieldTypes))
                ' Test if the current field contains a date.
                If IsDate(arrCurrentRecord(tmpCount)) Then
                    ' Specify the field type as a timestamp.
                    arrFieldTypes(tmpCount) = TYPE_TIMESTAMP
                ' Test if the current field contains a number.
                ElseIf IsNumeric(arrCurrentRecord(tmpCount)) Then
                    ' Test if the current field contains a decimal.
                    If InStr(arrCurrentRecord(tmpCount),".") Then
                        ' Specify the field type as a real number.
                        arrFieldTypes(tmpCount) = TYPE_REAL
                    Else
                        ' Specify the field type as an integer.
                        arrFieldTypes(tmpCount) = TYPE_INTEGER
                    End If
                ' Test if the current field is null.
                ElseIf IsNull(arrCurrentRecord(tmpCount)) Then
                    ' Specify the field type as NULL.
                    arrFieldTypes(tmpCount) = TYPE_NULL
                ' Test if the current field is empty.
                ElseIf IsEmpty(arrCurrentRecord(tmpCount)) Then
                    ' Specify the field type as NULL.
                    arrFieldTypes(tmpCount) = TYPE_NULL
                ' Otherwise, assume it's a string.
                Else
                    ' Specify the field type as a string.
                    arrFieldTypes(tmpCount) = TYPE_STRING
                End If
            Next
            ' Temporarily close the log file.
            objFile.Close
            ' Re-open the specified file.
            Set objFile = objFSO.OpenTextFile(strFileName,1,False)
            ' Test for a header row.
            If blnHeaderRow = True Then
                ' Skip the first row.
                objFile.SkipLine
            End If
            ' Return success status.
            OpenInput = 0
        Else
            ' Return a file not found status.
            OpenInput = 2
        End If
    End If
End Function

' --------------------------------------------------------------------------------
' Close the input session.
' --------------------------------------------------------------------------------

Public Function CloseInput(blnAbort)
    ' Free the objects.
    Set objFile = Nothing
    Set objFSO = Nothing
    ' Set a flag to indicate that the specified file is closed.
    blnFileOpen = False
End Function

' --------------------------------------------------------------------------------
' Return the count of fields.
' --------------------------------------------------------------------------------

Public Function GetFieldCount()
    ' Specify the default value.
    GetFieldCount = 0
    ' Test if a file is open.
    If (blnFileOpen = True) Then
        ' Test for the number of field names.
        If UBound(arrFieldNames) > 0 Then
            ' Return the count of fields.
            GetFieldCount = UBound(arrFieldNames) + 1
        End If
    End If
End Function

' --------------------------------------------------------------------------------
' Return the specified field's name.
' --------------------------------------------------------------------------------

Public Function GetFieldName(intFieldIndex)
    ' Specify the default value.
    GetFieldName = Null
    ' Test if a file is open.
    If (blnFileOpen = True) Then
        ' Test if the index is valid.
        If intFieldIndex<=UBound(arrFieldNames) Then
            ' Return the specified field name.
            GetFieldName = arrFieldNames(intFieldIndex)
        End If
    End If
End Function

' --------------------------------------------------------------------------------
' Return the specified field's type.
' --------------------------------------------------------------------------------

Public Function GetFieldType(intFieldIndex)
    ' Specify the default value.
    GetFieldType = Null
    ' Test if a file is open.
    If (blnFileOpen = True) Then
        ' Test if the index is valid.
        If intFieldIndex<=UBound(arrFieldTypes) Then
            ' Return the specified field type.
            GetFieldType = arrFieldTypes(intFieldIndex)
        End If
    End If
End Function

' --------------------------------------------------------------------------------
' Return the specified field's value.
' --------------------------------------------------------------------------------

Public Function GetValue(intFieldIndex)
    ' Specify the default value.
    GetValue = Null
    ' Test if a file is open.
    If (blnFileOpen = True) Then
        ' Test if the index is valid.
        If intFieldIndex<=UBound(arrCurrentRecord) Then
            ' Return the specified field value based on the field type.
            Select Case arrFieldTypes(intFieldIndex)
                Case TYPE_INTEGER:
                    GetValue = CInt(arrCurrentRecord(intFieldIndex))
                Case TYPE_REAL:
                    GetValue = CDbl(arrCurrentRecord(intFieldIndex))
                Case TYPE_STRING:
                    GetValue = CStr(arrCurrentRecord(intFieldIndex))
                Case TYPE_TIMESTAMP:
                    GetValue = CDate(arrCurrentRecord(intFieldIndex))
                Case Else
                    GetValue = Null
            End Select
        End If
    End If
End Function
  
' --------------------------------------------------------------------------------
' Read the next record, and return true or false if there is more data.
' --------------------------------------------------------------------------------

Public Function ReadRecord()
    ' Specify the default value.
    ReadRecord = False
    ' Test if a file is open.
    If (blnFileOpen = True) Then
        ' Test if there is more data.
        If objFile.AtEndOfStream Then
            ' Flag the log file as having no more data.
            ReadRecord = False
        Else
            ' Read the current record.
            arrCurrentRecord = Split(objFile.ReadLine,strSeparator)
            ' Flag the log file as having more data to process.
            ReadRecord = True
        End If
    End If
End Function

  </SCRIPT>

</SCRIPTLET>

After you have saved the scriptlet code to your computer, you register it by using the following syntax:

regsvr32 Generic.LogParser.Scriptlet.sct

At the very minimum, you can now use the COM plug-in with Log Parser by using syntax like the following:

logparser "SELECT * FROM 'C:\Foo\Bar.log'" -i:COM -iProgID:Generic.LogParser.Scriptlet

Next, let's analyze what this sample does.

Examining the Generic Scriptlet in Detail

Here are the different parts of the scriptlet and what they do:

  • The <registration> section of the scriptlet sets up the COM registration information; you'll notice the COM component class name and GUID, as well as version information and a general description. (Note that you should generate your own GUID for each scriptlet that you create.)
  • The <implements> section declares the public methods that the COM plug-in has to support.
  • The <script>section contains the actual implementation:
    • The first part of the script section declares the global variables that will be used:
      • The strSeparator  constant defines the delimiter that is used to separate the data between fields/columns in a text-based log file.
      • The blnHeaderRow  constant defines whether the first row in a text-based log file contains the names of the fields/columns:
        • If set to True, the plug-in will use the data in the first line of the log file to name the fields/columns.
        • If set to False, the plug-in will define generic field/column names like "Field1", "Field2", etc.
    • The second part of the script contains the required methods:
      • The OpenInput()  method performs several tasks:
        • Locates and opens the log file that you specify in your SQL statement, or returns an error if the log file cannot be found.
        • Determines the number, names, and data types of fields/columns in the log file.
      • The CloseInput()  method cleans up the session by closing the log file and destroying objects.
      • The GetFieldCount()  method returns the number of fields/columns in the log file.
      • The GetFieldName()  method returns the name of a field/column in the log file.
      • The GetFieldType()  method returns the data type of a field/column in the log file. As a reminder, Log Parser supports the following five data types for COM plug-ins: TYPE_INTEGER, TYPE_REAL, TYPE_STRING, TYPE_TIMESTAMP, and TYPE_NULL.
      • The GetValue()  method returns the data value of a field/column in the log file.
      • The ReadRecord()  method moves to the next line in the log file. This method returns True if there is additional data to read, or False when the end of data is reached.

Next, let's look at how to use the sample.

Using the Generic Scriptlet with Log Parser

As a sample log file for this blog, I'm going to use the data in the Sample XML File (books.xml) from MSDN. By running a quick Log Parser query that I will show later, I was able to export data from the XML file into text file named "books.log" that represents an example of a simple log file format that I have had to work with in the past:

id|publish_date|author|title|price
bk101|2000-10-01|Gambardella, Matthew|XML Developer's Guide|44.950000
bk102|2000-12-16|Ralls, Kim|Midnight Rain|5.950000
bk103|2000-11-17|Corets, Eva|Maeve Ascendant|5.950000
bk104|2001-03-10|Corets, Eva|Oberon's Legacy|5.950000
bk105|2001-09-10|Corets, Eva|The Sundered Grail|5.950000
bk106|2000-09-02|Randall, Cynthia|Lover Birds|4.950000
bk107|2000-11-02|Thurman, Paula|Splish Splash|4.950000
bk108|2000-12-06|Knorr, Stefan|Creepy Crawlies|4.950000
bk109|2000-11-02|Kress, Peter|Paradox Lost|6.950000
bk110|2000-12-09|O'Brien, Tim|Microsoft .NET: The Programming Bible|36.950000
bk111|2000-12-01|O'Brien, Tim|MSXML3: A Comprehensive Guide|36.950000
bk112|2001-04-16|Galos, Mike|Visual Studio 7: A Comprehensive Guide|49.950000

In this example, the data is pretty easy to understand - the first row contains the list of field/column names, and the fields/columns are separated by the pipe ("|") character throughout the log file. That being said, you could easily change my sample code to use a different delimiter that your custom log files use.

With that in mind, let's look at some Log Parser examples.

Example #1: Retrieving Data from a Custom Log

The first thing that you should try is to simply retrieve data from your custom plug-in, and the following query should serve as an example:

logparser "SELECT * FROM 'C:\sample\books.log'" -i:COM -iProgID:Generic.LogParser.Scriptlet

The above query will return results like the following:

idpublish_dateauthortitleprice
-----------------------------------------------------------------------------------------
bk101 10/1/2000 0:00:00 Gambardella, Matthew XML Developer's Guide 44.950000
bk102 12/16/2000 0:00:00 Ralls, Kim Midnight Rain 5.950000
bk103 11/17/2000 0:00:00 Corets, Eva Maeve Ascendant 5.950000
bk104 3/10/2001 0:00:00 Corets, Eva Oberon's Legacy 5.950000
bk105 9/10/2001 0:00:00 Corets, Eva The Sundered Grail 5.950000
bk106 9/2/2000 0:00:00 Randall, Cynthia Lover Birds 4.950000
bk107 11/2/2000 0:00:00 Thurman, Paula Splish Splash 4.950000
bk108 12/6/2000 0:00:00 Knorr, Stefan Creepy Crawlies 4.950000
bk109 11/2/2000 0:00:00 Kress, Peter Paradox Lost 6.950000
bk110 12/9/2000 0:00:00 O'Brien, Tim Microsoft .NET: The Programming Bible 36.950000
bk111 12/1/2000 0:00:00 O'Brien, Tim MSXML3: A Comprehensive Guide 36.950000
bk112 4/16/2001 0:00:00 Galos, Mike Visual Studio 7: A Comprehensive Guide 49.950000
         
Statistics:  
-----------  
Elements processed: 12
Elements output: 12
Execution time: 0.16 seconds

While the above example works a good proof-of-concept for functionality, it's not overly useful, so let's look at additional examples.

Example #2: Reformatting Log File Data

Once you have established that you can retrieve data from your custom plug-in, you can start taking advantage of Log Parser's features to process your log file data. In this example, I will use several of the built-in functions to reformat the data:

logparser "SELECT id AS ID, TO_DATE(publish_date) AS Date, author AS Author, SUBSTR(title,0,20) AS Title, STRCAT(TO_STRING(TO_INT(FLOOR(price))),SUBSTR(TO_STRING(price),INDEX_OF(TO_STRING(price),'.'),3)) AS Price FROM 'C:\sample\books.log'" -i:COM -iProgID:Generic.LogParser.Scriptlet

The above query will return results like the following:

IDDateAuthorTitlePrice
------------------------------------------------------------
bk101 10/1/2000 Gambardella, Matthew XML Developer's Guid 44.95
bk102 12/16/2000 Ralls, Kim Midnight Rain 5.95
bk103 11/17/2000 Corets, Eva Maeve Ascendant 5.95
bk104 3/10/2001 Corets, Eva Oberon's Legacy 5.95
bk105 9/10/2001 Corets, Eva The Sundered Grail 5.95
bk106 9/2/2000 Randall, Cynthia Lover Birds 4.95
bk107 11/2/2000 Thurman, Paula Splish Splash 4.95
bk108 12/6/2000 Knorr, Stefan Creepy Crawlies 4.95
bk109 11/2/2000 Kress, Peter Paradox Lost 6.95
bk110 12/9/2000 O'Brien, Tim Microsoft .NET: The 36.95
bk111 12/1/2000 O'Brien, Tim MSXML3: A Comprehens 36.95
bk112 4/16/2001 Galos, Mike Visual Studio 7: A C 49.95
         
Statistics:  
-----------  
Elements processed: 12
Elements output: 12
Execution time: 0.02 seconds

This example reformats the dates and prices a little nicer, and it truncates the book titles at 20 characters so they fit a little better on some screens.

Example #3: Processing Log File Data

In addition to simply reformatting your data, you can use Log Parser to group, sort, count, total, etc., your data. The following example illustrates how to use Log Parser to count the number of books by author in the log file:

logparser "SELECT author AS Author, COUNT(Title) AS Books FROM 'C:\sample\books.log' GROUP BY Author ORDER BY Author" -i:COM -iProgID:Generic.LogParser.Scriptlet

The above query will return results like the following:

AuthorBooks
-------------------------
Corets, Eva 3
Galos, Mike 1
Gambardella, Matthew 1
Knorr, Stefan 1
Kress, Peter 1
O'Brien, Tim 2
Ralls, Kim 1
Randall, Cynthia 1
Thurman, Paula 1
   
Statistics:  
-----------  
Elements processed: 12
Elements output: 9
Execution time: 0.03 seconds

The results are pretty straight-forward: Log Parser parses the data and presents you with a list of alphabetized authors and the total number of books that were written by each author.

Example #4: Creating Charts

You can also use data from your custom log file to create charts through Log Parser. If I modify the above example, all that I need to do is add a few parameters to create a chart:

logparser "SELECT author AS Author, COUNT(Title) AS Books INTO Authors.gif FROM 'C:\sample\books.log' GROUP BY Author ORDER BY Author" -i:COM -iProgID:Generic.LogParser.Scriptlet -fileType:GIF -groupSize:800x600 -chartType:Pie -categories:OFF -values:ON -legend:ON

The above query will create a chart like the following:

I admit that it's not a very pretty-looking chart - you can look at the other posts in my Log Parser series for some examples about making Log Parser charts more interesting.

Summary

In this blog post and my last post, I have illustrated a few examples that should help developers get started writing their own custom input format plug-ins for Log Parser. As I mentioned in each of the blog posts where I have used scriptlets for the COM objects, I would typically use C# or C++ to create a COM component, but using a scriptlet is much easier for demos because it doesn't require installing Visual Studio and compiling a DLL.

There is one last thing that I would like to mention before I finish this blog; I mentioned earlier that I had used Log Parser to reformat the sample Books.xml file into a generic log file that I could use for the examples in this blog. Since Log Parser supports XML as an input format and it allows you to customize your output, I wrote the following simple Log Parser query to reformat the XML data into a format that I had often seen used for text-based log files:

logparser.exe "SELECT id,publish_date,author,title,price INTO books.log FROM books.xml" -i:xml -o:tsv -headers:ON -oSeparator:"|"

Actually, this ability to change data formats is one of the hidden gems of Log Parser; I have often used Log Parser to change the data from one type of log file to another - usually so that a different program can access the data. For example, if you were given the log file with a pipe ("|") delimiter like I used as an example, you could easily use Log Parser to convert that data into the CSV format so you could open it in Excel:

logparser.exe "SELECT id,publish_date,author,title,price INTO books.csv FROM books.log" -i:tsv -o:csv -headers:ON -iSeparator:"|" -oDQuotes:on

I hope these past few blog posts help you to get started writing your own custom input format plug-ins for Log Parser.

That's all for now. ;-)

Note: This blog was originally posted at http://blogs.msdn.com/robert_mcmurray/

Posted: Feb 28 2013, 10:34 by Bob | Comments (0)
  • Currently 0/5 Stars.
  • 1
  • 2
  • 3
  • 4
  • 5
Filed under: LogParser | Scripting | XML
Social Bookmarks: E-mail | Kick it! | DZone it! | del.icio.us

Advanced Log Parser Part 6 - Creating a Simple Custom Input Format Plug-In

In Part 4 of this series, I illustrated how to create a new COM-based input provider for Log Parser from a custom input format:

Advanced Log Parser Charts Part 4 - Adding Custom Input Formats

For the sample that I published in that blog, I wrote a plug-in that consumed FTP RSCA events, which is highly structured data, and it added a lot of complexity to my example. In the past ten months or so since I published my original blog, I've had several requests for additional information about how to get started writing COM-based input formats for Log Parser, so it occurred to me that perhaps I could have shown a simpler example to get people started instead of diving straight into parsing RSCA data. ;-)

With that in mind, I thought that I would write a couple of blog posts with simpler examples to help anyone who wants to get started writing custom input formats for Log Parser.

For this blog post, I will show you how to create a very basic COM-based input format provider for Log Parser that simply returns static data; you could use this sample as a template to quickly get up-and-running with the basic concepts. (I promise to follow this blog with another real-world example that is still easier-to-use than my RSCA example.)

A Reminder about Creating COM-based plug-ins for Log Parser

In the blog that I referred to earlier, I mentioned that a COM plug-in has to support the following public methods:

Method NameDescription
OpenInput Opens your data source and sets up any initial environment settings.
GetFieldCount Returns the number of fields that your plug-in will provide.
GetFieldName Returns the name of a specified field.
GetFieldType Returns the datatype of a specified field.
GetValue Returns the value of a specified field.
ReadRecord Reads the next record from your data source.
CloseInput Closes your data source and cleans up any environment settings.

Once you have created and registered a COM plug-in, you call it by using something like the following syntax:

logparser.exe "SELECT * FROM FOO" -i:COM -iProgID:BAR

In the preceding example, FOO is a data source that makes sense to your plug-in, and BAR is the COM class name for your plug-in.

Creating a Simple COM plug-in for Log Parser

Once again, I'm going to demonstrate how to create a COM component by using a scriptlet, which I like to use for demos because they are quick to design, they're easily portable, and updates take place immediately since no compilation is required. (All of that being said, if I were writing a real COM plug-in for Log Parser, I would use C# or C++.)

To create the sample COM plug-in, copy the following code into a text file, and save that file as "Simple.LogParser.Scriptlet.sct" to your computer. (Note: The *.SCT file extension tells Windows that this is a scriptlet file.)

<SCRIPTLET>
  <registration
    Description="Simple Log Parser Scriptlet"
    Progid="Simple.LogParser.Scriptlet"
    Classid="{4e616d65-6f6e-6d65-6973-526f62657274}"
    Version="1.00"
    Remotable="False" />
  <comment>
  EXAMPLE: logparser "SELECT * FROM FOOBAR" -i:COM -iProgID:Simple.LogParser.Scriptlet
  </comment>
  <implements id="Automation" type="Automation">
    <method name="OpenInput">
      <parameter name="strValue"/>
    </method>
    <method name="GetFieldCount" />
    <method name="GetFieldName">
      <parameter name="intFieldIndex"/>
    </method>
    <method name="GetFieldType">
      <parameter name="intFieldIndex"/>
    </method>
    <method name="ReadRecord" />
    <method name="GetValue">
      <parameter name="intFieldIndex"/>
    </method>
    <method name="CloseInput">
      <parameter name="blnAbort"/>
    </method>
  </implements>
  <SCRIPT LANGUAGE="VBScript">

Option Explicit

Const MAX_RECORDS = 5
Dim intRecordCount

' --------------------------------------------------------------------------------
' Open the input session.
' --------------------------------------------------------------------------------

Public Function OpenInput(strValue)
    intRecordCount = 0
End Function

' --------------------------------------------------------------------------------
' Close the input session.
' --------------------------------------------------------------------------------

Public Function CloseInput(blnAbort)
End Function

' --------------------------------------------------------------------------------
' Return the count of fields.
' --------------------------------------------------------------------------------

Public Function GetFieldCount()
    GetFieldCount = 5
End Function

' --------------------------------------------------------------------------------
' Return the specified field's name.
' --------------------------------------------------------------------------------

Public Function GetFieldName(intFieldIndex)
    Select Case CInt(intFieldIndex)
        Case 0:
            GetFieldName = "INTEGER"
        Case 1:
            GetFieldName = "REAL"
        Case 2:
            GetFieldName = "STRING"
        Case 3:
            GetFieldName = "TIMESTAMP"
        Case 4:
            GetFieldName = "NULL"
        Case Else
            GetFieldName = Null
    End Select
End Function

' --------------------------------------------------------------------------------
' Return the specified field's type.
' --------------------------------------------------------------------------------

Public Function GetFieldType(intFieldIndex)
    ' Define the field type constants.
    Const TYPE_INTEGER   = 1
    Const TYPE_REAL      = 2
    Const TYPE_STRING    = 3
    Const TYPE_TIMESTAMP = 4
    Const TYPE_NULL      = 5
    Select Case CInt(intFieldIndex)
        Case 0:
            GetFieldType = TYPE_INTEGER
        Case 1:
            GetFieldType = TYPE_REAL
        Case 2:
            GetFieldType = TYPE_STRING
        Case 3:
            GetFieldType = TYPE_TIMESTAMP
        Case 4:
            GetFieldType = TYPE_NULL
        Case Else
            GetFieldType = Null
    End Select
End Function

' --------------------------------------------------------------------------------
' Return the specified field's value.
' --------------------------------------------------------------------------------

Public Function GetValue(intFieldIndex)
    Select Case CInt(intFieldIndex)
        Case 0:
            GetValue = 1
        Case 1:
            GetValue = 1.0
        Case 2:
            GetValue = "One"
        Case 3:
            GetValue = Now
        Case Else
            GetValue = Null
    End Select
End Function
  
' --------------------------------------------------------------------------------
' Read the next record, and return true or false if there is more data.
' --------------------------------------------------------------------------------

Public Function ReadRecord()
    intRecordCount = intRecordCount + 1
    If intRecordCount <= MAX_RECORDS Then
        ReadRecord = True
    Else
        ReadRecord = False
    End If
End Function

  </SCRIPT>

</SCRIPTLET>

After you have saved the scriptlet code to your computer, you register it by using the following syntax:

regsvr32 Simple.LogParser.Scriptlet.sct

At the very minimum, you can now use the COM plug-in with Log Parser by using syntax like the following:

logparser "SELECT * FROM FOOBAR" -i:COM -iProgID:Simple.LogParser.Scriptlet

This will return results like the following:

INTEGERREALSTRINGTIMESTAMPNULL
-------------------------------------------
1 1.000000 One 2/26/2013 19:42:12 -
1 1.000000 One 2/26/2013 19:42:12 -
1 1.000000 One 2/26/2013 19:42:12 -
1 1.000000 One 2/26/2013 19:42:12 -
1 1.000000 One 2/26/2013 19:42:12 -
         
Statistics:        
-----------        
Elements processed: 5      
Elements output: 5      
Execution time: 0.01 seconds      

Next, let's analyze what this sample does.

Examining the Sample Scriptlet Contents in Detail

Here are the different parts of the scriptlet and what they do:

  • The <registration> section of the scriptlet sets up the COM registration information; you'll notice the COM component class name and GUID, as well as version information and a general description. (Note that you should generate your own GUID for each scriptlet that you create.)
  • The <implements> section declares the public methods that the COM plug-in has to support.
  • The <script>section contains the actual implementation:
    • The OpenInput() method opens your data source, although in this example it only initializes the record count. (Note that the value that is passed to the method will be ignored in this example.)
    • The CloseInput() method would normally clean up your session, (e.g. close a data file or database, etc.), but it doesn't do anything in this example.
    • The GetFieldCount() method returns the number of data fields in each record of your data, which is static in this example.
    • The GetFieldName() method returns the name of a field that is passed to the method as a number; the names are static in this example.
    • The GetFieldType() method returns the data type of a field that is passed to the method as a number, which are statically-defined in this example. As a reminder, Log Parser supports the following five data types for COM plug-ins: TYPE_INTEGER, TYPE_REAL, TYPE_STRING, TYPE_TIMESTAMP, and TYPE_NULL.
    • The GetValue() method returns the data value of a field that is passed to the method as a number. Once again, the data values are statically-defined in this example.
    • The ReadRecord() method moves to the next record in your data set; this method returns True if there is data to read, or False when the end of data is reached. In this example, the method increments the record counter and sets the status based on whether the maximum number of records has been reached.

Summary

That wraps up the simplest example that I could put together of a COM-based input provider for Log Parser. In my next blog, I'll show how to create a generic COM-based input provider for Log Parser that you can use to parse text-based log files.

Note: This blog was originally posted at http://blogs.msdn.com/robert_mcmurray/
Posted: Feb 26 2013, 17:38 by Bob | Comments (0)
  • Currently 0/5 Stars.
  • 1
  • 2
  • 3
  • 4
  • 5
Filed under: LogParser | Scripting
Social Bookmarks: E-mail | Kick it! | DZone it! | del.icio.us

Restarting the FTP Service Orphans a DLLHOST.EXE Process

I was recently creating a new authentication provider using FTP extensibility, and I ran into a weird behavior that I had seen before. With that in mind, I thought my situation would make a great blog subject because someone else may run into it.

Here are the details of the situation: let's say that you are developing a new FTP provider for IIS, and your code changes never seem to take effect. Your provider appears to be working, it's just that any new functionality is not reflected in your provider's behavior. You restart the FTP service as a troubleshooting step, but that does not appear to make any difference.

I'll bypass mentioning any other troubleshooting tasks and cut to the chase - if you read my Changing the Identity of the FTP 7 Extensibility Process blog post a year ago, you will recall that I mentioned that all custom FTP extensibility providers are executed through COM+ in a DLLHOST.exe process. When you restart the FTP service, that should clean up the DLLHOST.EXE process that is being used for FTP extensibility. However, if you are developing custom FTP providers and the DLLHOST.EXE process is not terminated by the FTP service, you may find yourself in a situation where you have a DLLHOST.EXE process in memory that contains an older copy of your provider, which will not be removed from memory until the DLLHOST.EXE process for FTP extensibility has been forcibly terminated.

If you have read some of my earlier blog posts or walkthroughs on IIS.NET, you may have noticed that I generally like to use a few pre-build and post-build commands in my FTP projects; usually I add these commands in order to to automatically register/unregister my FTP providers in the Global Assembly Cache (GAC).

With a little modification and some command-line wizardry, you can automate the termination of any orphaned DLLHOST.EXE processes that are being used for FTP extensibility. With that in mind, here are some example pre-build/post-build commands that will unregister/reregister your provider in the GAC, restart the FTP service, and terminate any orphaned FTP extensibility DLLHOST.EXE processes.

Note: The following syntax was written using Visual Studio 2010; you would need to change "%VS100COMNTOOLS%" to "%VS90COMNTOOLS%" for Visual Studio 2008 or "%VS110COMNTOOLS%" for Visual Studio 2012.

Pre-build Commands:

net stop ftpsvc

call "%VS100COMNTOOLS%\vsvars32.bat">nul

cd /d "$(TargetDir)"

gacutil.exe /uf "$(TargetName)"

for /f "usebackq tokens=1,2* delims=," %%a in (`tasklist /fi "MODULES eq Microsoft.Web.FtpServer.*" /fi "IMAGENAME eq DLLHOST.EXE" /fo csv ^| find /i "dllhost.exe"`) do taskkill /f /pid %%b

Post-build Commands:

call "%VS100COMNTOOLS%\vsvars32.bat">nul

gacutil.exe /if "$(TargetPath)"

net start ftpsvc

The syntax is a little tricky for the FOR statement, so be carefully when typing or copying/pasting that into your projects. For example, you need to make sure that all of the code from the FOR statement through the TASKKILL command are on the same line in your project's properties.

When you compile your provider, Visual Studio should display something like the following:

------ Rebuild All started: Project: FtpBlogEngineNetAuthentication, Configuration: Release Any CPU ------
The Microsoft FTP Service service is stopping.
The Microsoft FTP Service service was stopped successfully.

Microsoft (R) .NET Global Assembly Cache Utility. Version 4.0.30319.1
Copyright (c) Microsoft Corporation. All rights reserved.

Assembly: FtpBlogEngineNetAuthentication, Version=1.0.0.0, Culture=neutral, PublicKeyToken=426f62526f636b73, processorArchitecture=MSIL
Uninstalled: FtpBlogEngineNetAuthentication, Version=1.0.0.0, Culture=neutral, PublicKeyToken=426f62526f636b73, processorArchitecture=MSIL
Number of assemblies uninstalled = 1
Number of failures = 0
SUCCESS: The process with PID 12656 has been terminated.
FtpBlogEngineNetAuthentication -> C:\Users\dude\Documents\Visual Studio 2010\Projects\FtpBlogEngineNetAuthentication\FtpBlogEngineNetAuthentication\bin\Release\FtpBlogEngineNetAuthentication.dll
Microsoft (R) .NET Global Assembly Cache Utility. Version 4.0.30319.1
Copyright (c) Microsoft Corporation. All rights reserved.

Assembly successfully added to the cache
The Microsoft FTP Service service is starting.
The Microsoft FTP Service service was started successfully.

========== Rebuild All: 1 succeeded, 0 failed, 0 skipped ==========

If you analyze the output from the build process, you will see that the commands in my earlier samples stopped the FTP service, removed the existing assembly from the GAC, terminated any orphaned DLLHOST.EXE processes, registered the newly-built DLL in the GAC, and then restarted the FTP service.

By utilizing these pre-build/post-build commands, I have been able to work around situations where a DLLHOST.EXE process is being orphaned and caching old assemblies in memory.

Note: This blog was originally posted at http://blogs.msdn.com/robert_mcmurray/
Posted: Jan 30 2013, 18:13 by Bob | Comments (0)
  • Currently 0/5 Stars.
  • 1
  • 2
  • 3
  • 4
  • 5
Filed under: Scripting | FTP | Extensibility
Social Bookmarks: E-mail | Kick it! | DZone it! | del.icio.us

Programmatically Starting and Stopping FTP Sites in IIS 7 and IIS 8

I was recently contacted by someone who was trying to use Windows Management Instrumentation (WMI) code to stop and restart FTP websites by using code that he had written for IIS 6.0; his code was something similar to the following:

Option Explicit
On Error Resume Next

Dim objWMIService, colItems, objItem

' Attach to the IIS service.
Set objWMIService = GetObject("winmgmts:\root\microsoftiisv2")
' Retrieve the collection of FTP sites.
Set colItems = objWMIService.ExecQuery("Select * from IIsFtpServer")
' Loop through the sites collection.
For Each objItem in colItems
    ' Restart one single website.
    If (objItem.Name = "MSFTPSVC/1") Then
        Err.Clear
        objItem.Stop
        If (Err.Number <> 0) Then WScript.Echo Err.Number
        objItem.Start
        If (Err.Number <> 0) Then WScript.Echo Err.Number
    End If
Next

The problem that the customer was seeing is that this query did not return the list of FTP-based websites for IIS 7.0 or IIS 7.5 (called IIS7 henceforth), although changing the class in the query from IIsFtpServer to IIsWebServer would make the script work with HTTP-based websites those versions of IIS7.

The problem with the customer's code was that he is using WMI to manage IIS7; this relies on our old management APIs that have been deprecated, although part of that model is partially available through the metabase compatibility feature in IIS7. Here's what I mean by "partially": only a portion of the old ADSI/WMI objects are available, and unfortunately FTP is not part of the objects that can be scripted through the metabase compatibility feature in IIS7.

That being said, what the customer wants to do is still possible through scripting in both IIS7 and IIS8, and the following sample shows how to loop through all of the sites, determine which sites have FTP bindings, and then stop/start FTP for each site. To use this script, copy the code into a text editor like Windows Notepad and save it with a name like "RestartAllFtpSites.vbs" to your system, then double-click the file to run it.

' Temporarily disable breaking on runtime errors.
On Error Resume Next

' Create an Admin Manager object.
Set adminManager = CreateObject("Microsoft.ApplicationHost.AdminManager")
adminManager.CommitPath = "MACHINE/WEBROOT/APPHOST"

' Test for commit path support.
If Err.Number <> 0 Then
    Err.Clear
    ' Create a Writable Admin Manager object.
    Set adminManager = CreateObject("Microsoft.ApplicationHost.WritableAdminManager")
    adminManager.CommitPath = "MACHINE/WEBROOT/APPHOST"
    If Err.Number <> 0 Then WScript.Quit
End If

' Resume breaking on runtime errors.
On Error Goto 0

' Retrieve the sites collection.
Set sitesSection = adminManager.GetAdminSection("system.applicationHost/sites", "MACHINE/WEBROOT/APPHOST")
Set sitesCollection = sitesSection.Collection

' Loop through the sites collection.
For siteCount = 0 To CInt(sitesCollection.Count)-1
    isFtpSite = False
    ' Determine if the current site is an FTP site by checking the bindings.
    Set siteElement = sitesCollection(siteCount)
    Set bindingsCollection = siteElement.ChildElements.Item("bindings").Collection
    For bindingsCount = 0 To CInt(bindingsCollection.Count)-1
        Set bindingElement = bindingsCollection(bindingsCount)
        If StrComp(CStr(bindingElement.Properties.Item("protocol").Value),"ftp",vbTextCompare)=0 Then
            isFtpSite = True
            Exit For
        End If
    Next
    ' If it's an FTP site, start and stop the site.
    If isFtpSite = True Then
        Set ftpServerElement = siteElement.ChildElements.Item("ftpServer")
        ' Create an instance of the Stop method.
        Set stopFtpSite = ftpServerElement.Methods.Item("Stop").CreateInstance()
        ' Execute the method to stop the FTP site.
        stopFtpSite.Execute()
        ' Create an instance of the Start method.
        Set startFtpSite = ftpServerElement.Methods.Item("Start").CreateInstance()
        ' Execute the method to start the FTP site.
        startFtpSite.Execute()
    End If
Next

And the following code sample shows how to stop/start a single FTP site. To use this script, copy the code into a text editor like Windows Notepad, rename the site name appropriately for one of your FTP sites, save it with a name like "RestartContosoFtpSite.vbs" to your system, then double-click the file to run it.

' Temporarily disable breaking on runtime errors.
On Error Resume Next

' Create an Admin Manager object.
Set adminManager = CreateObject("Microsoft.ApplicationHost.AdminManager")
adminManager.CommitPath = "MACHINE/WEBROOT/APPHOST"

' Test for commit path support.
If Err.Number <> 0 Then
    Err.Clear
    ' Create a Writable Admin Manager object.
    Set adminManager = CreateObject("Microsoft.ApplicationHost.WritableAdminManager")
    adminManager.CommitPath = "MACHINE/WEBROOT/APPHOST"
    If Err.Number <> 0 Then WScript.Quit
End If

' Resume breaking on runtime errors.
On Error Goto 0

' Retrieve the sites collection.
Set sitesSection = adminManager.GetAdminSection("system.applicationHost/sites", "MACHINE/WEBROOT/APPHOST")
Set sitesCollection = sitesSection.Collection

' Locate a specific site.
siteElementPos = FindElement(sitesCollection, "site", Array("name", "ftp.contoso.com"))
If siteElementPos = -1 Then
    WScript.Echo "Site was not found!"
    WScript.Quit
End If

' Determine if the selected site is an FTP site by checking the bindings.
Set siteElement = sitesCollection(siteElementPos)
Set bindingsCollection = siteElement.ChildElements.Item("bindings").Collection
For bindingsCount = 0 To CInt(bindingsCollection.Count)-1
    Set bindingElement = bindingsCollection(bindingsCount)
    If StrComp(CStr(bindingElement.Properties.Item("protocol").Value),"ftp",vbTextCompare)=0 Then
        isFtpSite = True
        Exit For
    End If
Next

' If it's an FTP site, start and stop the site.
If isFtpSite = True Then
    Set ftpServerElement = siteElement.ChildElements.Item("ftpServer")
    ' Create an instance of the Stop method.
    Set stopFtpSite = ftpServerElement.Methods.Item("Stop").CreateInstance()
    ' Execute the method to stop the FTP site.
    stopFtpSite.Execute()
    ' Create an instance of the Start method.
    Set startFtpSite = ftpServerElement.Methods.Item("Start").CreateInstance()
    ' Execute the method to start the FTP site.
    startFtpSite.Execute()
End If

' Locate and return the index for a specific element in a collection.
Function FindElement(collection, elementTagName, valuesToMatch)
   For i = 0 To CInt(collection.Count) - 1
      Set elem = collection.Item(i)
      If elem.Name = elementTagName Then
         matches = True
         For iVal = 0 To UBound(valuesToMatch) Step 2
            Set prop = elem.GetPropertyByName(valuesToMatch(iVal))
            value = prop.Value
            If Not IsNull(value) Then
               value = CStr(value)
            End If
            If Not value = CStr(valuesToMatch(iVal + 1)) Then
               matches = False
               Exit For
            End If
         Next
         If matches Then
            Exit For
         End If
      End If
   Next
   If matches Then
      FindElement = i
   Else
      FindElement = -1
   End If
End Function

I hope this helps!

Note: This blog was originally posted at http://blogs.msdn.com/robert_mcmurray/
Posted: Oct 03 2012, 08:57 by Bob | Comments (0)
  • Currently 0/5 Stars.
  • 1
  • 2
  • 3
  • 4
  • 5
Tags: , , ,
Social Bookmarks: E-mail | Kick it! | DZone it! | del.icio.us

Advanced Log Parser Charts Part 5 - Creating a Generic Chart Color Script

In Part 5 of this series, I'll show you how to create a generic script that you can use to add some color to your Log Parser charts. As I mentioned in Part 1 of this series, the default colors for Log parser charts are really dull and boring. For example, if I parse one month's worth of log files from one of my low-volume websites with the following query:

logparser.exe "SELECT date,COUNT(*) AS Hits INTO HITS.gif FROM *.log GROUP BY date ORDER BY date" -i:w3c -o:CHART -chartType:ColumnClustered -chartTitle:"" -q:ON

Log Parser will create the following ugly daily hits chart:

Generic Color Change Script

Here's the background story for this blog: I have a collection of scripts that I use to format my charts, several of which have faithfully served as the fodder for this blog series. With that in mind, I had a situation recently where I was querying logs with a series of data just like this, and of course the resulting charts were kind of hideous to look at. In one of the scripts that I often use, I create an array of colors to use, and then I apply the various colors to the individual data points in the series.

In the past I have always hard-coded the length for the array of colors based on the data that I am working with, but in this situation I had no idea how many data points I would have, so I decided to put together a quick script with an array that would work with a series of any size.

Here's the resulting script:

// Set a default color for the chart's data.
chart.SeriesCollection(0).Interior.Color = "#ffcccc";

// Define a short array of colors.
var colors = [
    "#ffff99", "#ff99ff", "#ff9999",
    "#99ffff", "#99ff99", "#9999ff",
    "#ffffcc", "#ffccff", "#ffcccc",
    "#ccffff", "#ccffcc", "#ccccff"
];

// Loop through the data points in the series.
for (x=0;x<chart.SeriesCollection(0).Points.Count;++x)
{
    // Set the color for the data point based on modulo division of the array length.
    chart.SeriesCollection(0).Points(x).Interior.Color = colors[x % colors.length ];
}

That's all that there is to the script - it's pretty simple. If I take the above script and save it as "FormatChart.js", I can use that script with my Log Parser query from earlier by adding an extra parameter to the command:

logparser.exe "SELECT date,COUNT(*) AS Hits INTO HITS.gif FROM *.log GROUP BY date ORDER BY date" -i:w3c -o:CHART -chartType:ColumnClustered -chartTitle:"" -q:ON -config:FormatChart.js

Now Log Parser will create the following daily hits chart with a great deal more color to it:

Okay - perhaps that's not the best color palette, but you get the idea. It looks even better when I change the query to use 3D charts:

logparser.exe "SELECT date,COUNT(*) AS Hits INTO HITS.gif FROM *.log GROUP BY date ORDER BY date" -i:w3c -o:CHART -chartType:Column3D -chartTitle:"" -q:ON -config:FormatChart.js

The above query creates the following chart:

Color Changing Pie Charts

I'd like to make a quick change to the script in order to make it work a little better with a pie chart:

// Set a default color for the chart's data.
chart.SeriesCollection(0).Interior.Color = "#cccccc";

// Define a short array of colors.
var colors = [
    "#cc3333", "#3333cc", "#33cc33",
    "#33cccc", "#cccc33", "#cc33cc"
];

// Loop through the data points in the series.
for (x=0;x<chart.SeriesCollection(0).Points.Count;++x)
{
    // Set the color for the data point based on modulo division of the array length.
    chart.SeriesCollection(0).Points(x).Interior.Color = colors[x % colors.length ];
}
// Rotate the chart 180 degrees - just so it looks a little better.
chartSpace.Charts(0).PlotArea.RotateClockwise();
chartSpace.Charts(0).PlotArea.RotateClockwise();

For this query I'd like to see a break down by HTTP status, and this necessitates some small change to the Log parser query:

logparser.exe "SELECT sc-status AS Status,COUNT(*) AS Hits INTO HITS.gif FROM *.log GROUP BY Status ORDER BY Status" -i:w3c -o:CHART -chartType:PieExploded3D -chartTitle:"" -q:ON -config:FormatChart.js

The above query creates the following chart:

Summary

That wraps it up for this blog - I hope that I've given you some ideas for ways that you can easily add some colors to some dull-looking Log Parser charts.

Note: This blog was originally posted at http://blogs.msdn.com/robert_mcmurray/
Posted: Aug 24 2012, 18:53 by Bob | Comments (0)
  • Currently 0/5 Stars.
  • 1
  • 2
  • 3
  • 4
  • 5
Filed under: LogParser | Scripting
Social Bookmarks: E-mail | Kick it! | DZone it! | del.icio.us

Creating "Pretty" XML using XSL and VBScript

I was working with an application recently that stored all of its settings in a large XML file, however, when I opened the XML in Windows Notepad, all I saw was a large blob of tags and text - there was no structured formatting to the XML, and that made it very difficult to change some of settings by hand. (Okay - I realize that some of you are probably thinking to yourselves, maybe I wasn't supposed to be editing those settings by hand - but that's just the way I do things around here... if I can't customize every setting to my heart's content, then it's just not worth using.)

In any event, I'll give you an example of what I mean by using the example XML database that's provided on MSDN at the following URL:

http://msdn.microsoft.com/en-us/library/windows/desktop/ms762271.aspx

Note - the entire XML file would be too long to repost here, so I'll just include an unstructured except from that file that resembles what my other XML looked like when I opened the file in Windows Notepad:

<?xml version="1.0"?><catalog><book id="bk101"><author>Gambardella, Matthew</author><title>XML Developer's Guide</title><genre>Computer</genre><price>44.95</price><publish_date>2000-10-01</publish_date><description>An in-depth look at creating applications with XML.</description></book><book id="bk102"><author>Ralls, Kim</author><title>Midnight Rain</title><genre>Fantasy</genre><price>5.95</price><publish_date>2000-12-16</publish_date><description>A former architect battles corporate zombies, an evil sorceress, and her own childhood to become queen of the world.</description></book><book id="bk103"><author>Corets, Eva</author><title>Maeve Ascendant</title><genre>Fantasy</genre><price>5.95</price><publish_date>2000-11-17</publish_date><description>After the collapse of a nanotechnology society in England, the young survivors lay the foundation for a new society.</description></book></catalog>

This is obviously difficult to read, and even more so when you are dealing with hundreds or thousands of lines of XML code. What would be considerably easier to read and edit would be something more like the following example:

<?xml version="1.0"?>
<catalog>
  <book id="bk101">
    <author>Gambardella, Matthew</author>
    <title>XML Developer's Guide</title>
    <genre>Computer</genre>
    <price>44.95</price>
    <publish_date>2000-10-01</publish_date>
    <description>An in-depth look at creating applications with XML.</description>
  </book>
  <book id="bk102">
    <author>Ralls, Kim</author>
    <title>Midnight Rain</title>
    <genre>Fantasy</genre>
    <price>5.95</price>
    <publish_date>2000-12-16</publish_date>
    <description>A former architect battles corporate zombies, an evil sorceress, and her own childhood to become queen of the world.</description>
  </book>
  <book id="bk103">
    <author>Corets, Eva</author>
    <title>Maeve Ascendant</title>
    <genre>Fantasy</genre>
    <price>5.95</price>
    <publish_date>2000-11-17</publish_date>
    <description>After the collapse of a nanotechnology society in England, the young survivors lay the foundation for a new society.</description>
  </book>
</catalog>

I had written a "Pretty XML" script sometime around ten years ago that read an XML file, collapsed all of the whitespace between tags, and then inserted CRLF sequences and TAB characters in order to reformat the file. This script worked great for many years, but I decided that it would be more advantageous to use XSL to transform the XML. (e.g. "Why continue to do things the hard way when you really don't need to?");-]

With that in mind, I rewrote my old script as the following example:

' ****************************************
' MAKE PRETTY XML
' ****************************************

Option Explicit

Const strInputFile = "InputFile.xml"
Const strOutputFile = "OutputFile.xml"

' ****************************************

Dim objInputFile, objOutputFile, strXML
Dim objFSO : Set objFSO = WScript.CreateObject("Scripting.FileSystemObject")
Dim objXML : Set objXML = WScript.CreateObject("Msxml2.DOMDocument")
Dim objXSL : Set objXSL = WScript.CreateObject("Msxml2.DOMDocument")

' ****************************************
' Put whitespace between tags. (Required for XSL transformation.)
' ****************************************

Set objInputFile = objFSO.OpenTextFile(strInputFile,1,False,-2)
Set objOutputFile = objFSO.CreateTextFile(strOutputFile,True,False)
strXML = objInputFile.ReadAll
strXML = Replace(strXML,"><",">" & vbCrLf & "<")
objOutputFile.Write strXML
objInputFile.Close
objOutputFile.Close

' ****************************************
' Create an XSL stylesheet for transformation.
' ****************************************

Dim strStylesheet : strStylesheet = _
"<xsl:stylesheet version=""1.0"" xmlns:xsl=""http://www.w3.org/1999/XSL/Transform"">" & _
"<xsl:output method=""xml"" indent=""yes""/>" & _
"<xsl:template match=""/"">" & _
"<xsl:copy-of select="".""/>" & _
"</xsl:template>" & _
"</xsl:stylesheet>"

' ****************************************
' Transform the XML.
' ****************************************

objXSL.loadXML strStylesheet
objXML.load strOutputFile
objXML.transformNode objXSL
objXML.save strOutputFile

WScript.Quit

This script is really straightforward in what it does:

  1. Creates two MSXML DOM Document objects:
    • One for XML
    • One for XSL
  2. Creates two file objects:
    • One for the input/source XML file
    • One for the output/destination XML
  3. Reads all of the source XML from the input file.
  4. Inserts whitespace between all of the XML tags in the source XML; this is required or the XSL transformation will not work properly.
  5. Saves the resulting XML into the output XML file.
  6. Dynamically creates a simple XSL file that will be used for transformation in one of the MSXML DOM Document objects.
  7. Loads the output XML file from earlier into the other MSXML DOM Document object.
  8. Transforms the source XML into well-formatted ("pretty") XML.
  9. Replaces the XML in the output file with the transformed XML.

That's all that there is to it.

Note: For more information about the XSL stylesheet that I used, see http://www.w3.org/TR/xslt.

Note: This blog was originally posted at http://blogs.msdn.com/robert_mcmurray/

Posted: Jul 06 2012, 06:07 by Bob | Comments (0)
  • Currently 0/5 Stars.
  • 1
  • 2
  • 3
  • 4
  • 5
Filed under: Scripting | XML
Tags: , ,
Social Bookmarks: E-mail | Kick it! | DZone it! | del.icio.us

Working with the Different IIS Express Modes and HTTPS

I had another great question from a customer the other day, and I thought that his question was the perfect impetus for me to write blog that explained the different modes of IIS Express.

The customer's issue was that he was trying to run IIS Express from a command-line by specifying the path to a folder and he wanted to use that with SSL. He couldn't find a way to accomplish that, so he asked Scott Hanselman if there was a switch that he was missing, and Scott sent him my way. In the meantime, he was copying one of the IIS Express template ApplicationHost.config files and configuring SSL by modifying the XML programmatically.

First of all, the short answer is that there isn't some form of "/https" switch for IIS Express that the customer was asking about.

But that being said, this seemed like a great occasion for me to explain a little bit of design architecture for IIS Express, which might help everyone understand a little bit about what's going on behind the scenes when you run IIS Express.

In case you weren't aware, there are actually two modes that you can use with IIS Express:

  • Personal Web Server Mode
  • Application Server Mode

Having said that, I'll explain what both of those fancy titles actually mean, and how you can use IIS Express with SSL.

Personal Web Server Mode

When you are using Personal Web Server Mode, one ApplicationHost.config file is created per user by default, (unless an alternate file is specified on the command-line), and by default that ApplicationHost.config file is kept in your "%UserProfile%\Documents\IISExpress\config" folder.

In this mode, websites are persistent like they are with the full version of IIS, and the template that is used to create the per-user ApplicationHost.config file is located at:

"%ProgramFiles%\IIS Express\config\templates\PersonalWebServer\ApplicationHost.config"

Note: When you are using Personal Web Server Mode, your default website is named "WebSite1".

The general syntax for Personal Web Server Mode is:

iisexpress.exe [/config:config-file] [/site:site-name] [/systray:true|false] [/siteid:site-id] [/userhome:user-home]

If you are using IIS Express from a command-line with no parameters, or you are using IIS Express with WebMatrix or Visual Studio, then you are using Personal Web Server Mode. You can use SSL by enabling HTTPS in either WebMatrix or Visual Studio, or you can modify your ApplicationHost.config file directly and add an HTTPS binding to a website.

Application Server Mode

When you are using "Application Server Mode," a temporary ApplicationHost.config file generated when IIS Express starts in the user's "%TEMP%\iisexpress" folder.

In this mode, sites are transient like they are with Cassini, and the template that is used to create the temporary ApplicationHost.config file is located at:

"%ProgramFiles%\IIS Express\AppServer\ApplicationHost.config"

Note: When you are using Application Server Mode, your default website is named "Development Web Site".

The general syntax for Application Server Mode is:

iisexpress.exe /path:app-path [/port:port-number] [/clr:clr-version] [/systray:true|false]

If you are using IIS Express from a command-line by specifying the path to a folder, then you are using Application Server Mode, and unfortunately you can't use SSL with this mode.

Using SSL with IIS Express

As I have already mentioned, if you are using Personal Web Server Mode, you can use SSL by enabling HTTPS in WebMatrix or Visual Studio if you are using either of those tools, or you can modify your ApplicationHost.config file directly and add an HTTPS binding to a website.

However, there is no way to specify HTTPS for Application Server Mode; but that being said, there are definitely workarounds that you can use.

Copying the template file like the customer was doing is a good place to start. But I need to state an important warning: you should never modify the actual template files that are installed with IIS Express! However, if you copy the template files somewhere else on your system, you can modify the copied files as much as you want.

If you are using IIS 8 Express, we've made it possible to use AppCmd.exe with any ApplicationHost.config file by using the "/apphostconfig" switch. So instead of modifying the XML directly, you can use AppCmd.exe to make your changes for you.

For example, the following batch file creates a temporary website and sets it up for use with HTTPS:

@echo off

pushd "%~dp0"

REM Create the website's folders.

md %SystemDrive%\myhttpstemp
md %SystemDrive%\myhttpstemp\wwwroot
md %SystemDrive%\myhttpstemp\config

REM Copy the template configuration file.

copy "%ProgramFiles%\IIS Express\AppServer\ApplicationHost.config" %SystemDrive%\myhttpstemp\config

REM Configure the website's home directory.

"%ProgramFiles%\IIS Express\appcmd.exe" set config -section:system.ApplicationHost/sites /"[name='Development Web Site'].[path='/'].[path='/'].physicalPath:%SystemDrive%\myhttpstemp\wwwroot" /commit:apphost /apphostconfig:%SystemDrive%\myhttpstemp\config\ApplicationHost.config

REM Configure the website for SSL.

"%ProgramFiles%\IIS Express\appcmd.exe" set config -section:system.ApplicationHost/sites /+"[name='Development Web Site'].bindings.[protocol='https',bindingInformation='127.0.0.1:8443:']" /commit:apphost /apphostconfig:%SystemDrive%\myhttpstemp\config\ApplicationHost.config

REM Enable directory browsing so this example works without a home page.

"%ProgramFiles%\IIS Express\appcmd.exe" set config "Development Web Site" -section:system.webServer/directoryBrowse /enabled:"True" /commit:apphost /apphostconfig:%SystemDrive%\myhttpstemp\config\ApplicationHost.config

REM Run the website with IIS Express.

"%ProgramFiles%\IIS Express\iisexpress.exe" /config:%SystemDrive%\myhttpstemp\config\ApplicationHost.config /siteid:1 /systray:false

REM Clean up the website folders.

rd /q /s %SystemDrive%\myhttpstemp

popd

As you can see in the above example, this is a little more involved than simply invoking Application Server Mode with a switch to enable HTTPS, but it's still very easy to do. The changes that we've made in IIS 8 Express make it easy to script Personal Web Server Mode in order to enable SSL for a temporary website.

In Closing...

I hope this information makes using the various IIS Express modes and SSL a little clearer, and you can get IIS 8 Express by following the link in the following blog post:

http://blogs.msdn.com/b/robert_mcmurray/archive/2012/05/31/microsoft-iis-8-0-express-release-candidate-is-released.aspx

Note: This blog was originally posted at http://blogs.msdn.com/robert_mcmurray/

Posted: Jul 03 2012, 07:06 by Bob | Comments (0)
  • Currently 0/5 Stars.
  • 1
  • 2
  • 3
  • 4
  • 5
Filed under: IIS Express | Scripting
Social Bookmarks: E-mail | Kick it! | DZone it! | del.icio.us

Advanced Log Parser Charts Part 4 - Adding Custom Input Formats

In Part 4 of this series, I'll show you how you can do a couple of cool things:

  • Create a new input format for Log Parser (and I'll use FTP RSCA data as an example)
  • Create charts from your custom input format

For the data source for my custom plug-in, I thought that it would be cool to consume the data from FTP 7's Runtime Status (RSCA). If you've followed some of my old blogs, you would have seen that around five years ago I wrote the following pair of blogs about programmatically viewing FTP 7 sessions:

I'm going to recycle some of the FTP RSCA concepts from those blogs in order to create my COM plug-in.

Log Parser Input Formats

If you're like me, you already realize that the existing features of Log Parser simply rock. But what most people don't realize is that Log Parser lets you extend the functionality by adding new input formats, so you can consume the data from any place where you feel compelled to sit down and write your own Log Parser module.

As a quick reminder, Log Parser supports the following built-in input formats:

  • IIS Log File Input Formats
    • IISW3C: parses IIS log files in the W3C Extended Log File Format.
    • IIS: parses IIS log files in the Microsoft IIS Log File Format.
    • BIN: parses IIS log files in the Centralized Binary Log File Format.
    • IISODBC: returns database records from the tables logged to by IIS when configured to log in the ODBC Log Format.
    • HTTPERR: parses HTTP error log files generated by Http.sys.
    • URLSCAN: parses log files generated by the URLScan IIS filter.
  • Generic Text File Input Formats
    • CSV: parses comma-separated values text files.
    • TSV: parses tab-separated and space-separated values text files.
    • XML: parses XML text files.
    • W3C: parses text files in the W3C Extended Log File Format.
    • NCSA: parses web server log files in the NCSA Common, Combined, and Extended Log File Formats.
    • TEXTLINE: returns lines from generic text files.
    • TEXTWORD: returns words from generic text files.
  • System Information Input Formats
    • EVT: returns events from the Windows Event Log and from Event Log backup files (.evt files).
    • FS: returns information on files and directories.
    • REG: returns information on registry values.
    • ADS: returns information on Active Directory objects.
  • Special-purpose Input Formats
    • NETMON: parses network capture files created by NetMon.
    • ETW: parses Enterprise Tracing for Windows trace log files and live sessions.
    • COM: provides an interface to Custom Input Format COM plug-ins.

This last input format, COM, is how you interface with Log Parser in order to create your own input formats. When you install Log Parser, there are a few COM-based samples in the Log Parser directory, and you can take a look at those when you get the chance.

A Brief Introduction to Creating a COM plug-ins for Log Parser

To start with, your COM plug-in has to support a few public methods - and each of these will be more clear when I create my plug-in later:

Method Name Description
OpenInput Opens your data source and sets up any initial environment settings.
GetFieldCount Returns the number of fields that your plug-in will provide.
GetFieldName Returns the name of a specified field.
GetFieldType Returns the datatype of a specified field.
GetValue Returns the value of a specified field.
ReadRecord Reads the next record from your data source.
CloseInput Closes your data source and cleans up any environment settings.

After you've created and registered your COM plug-in, you will call it by using something like the following syntax:

logparser "SELECT * FROM FOO" -i:COM -iProgID:BAR

In this example, FOO is some data source that makes sense to your plug-in, and BAR is the COM class name for your plug-in.

Creating the COM plug-in for FTP RSCA Data

I'm going to demonstrate how to create a COM component as a scriptlet, and then I'll call that from Log Parser to process the data. I chose to use a scriptlet for this demo because they are quick to design and they're easily portable. Since no compilation is required, updates take place on the fly. All of that being said, if I were writing a real COM plug-in for Log Parser, I would use C# or C++.

To create the sample COM plug-in, copy the following code into a text file, and save that file as "MSUtil.LogQuery.FtpRscaScriptlet.sct" to your computer. (Note: The *.SCT file extension tells Windows that this is a scriptlet file.)

<SCRIPTLET>
  <registration
    Description="FTP RSCA for Log Parser Scriptlet"
    Progid="MSUtil.LogQuery.FtpRscaScriptlet"
    Classid="{4e616d65-6f6e-6d65-6973-526f62657274}"
    Version="1.00"
    Remotable="False" />
  <comment>
  EXAMPLE 1: logparser "SELECT * FROM ftp.example.com" -i:COM -iProgID:MSUtil.LogQuery.FtpRscaScriptlet
  EXAMPLE 2: logparser "SELECT * FROM 1" -i:COM -iProgID:MSUtil.LogQuery.FtpRscaScriptlet
  </comment>
  <implements id="Automation" type="Automation">
    <method name="OpenInput">
      <parameter name="strValue"/>
    </method>
    <method name="GetFieldCount" />
    <method name="GetFieldName">
      <parameter name="intFieldIndex"/>
    </method>
    <method name="GetFieldType">
      <parameter name="intFieldIndex"/>
    </method>
    <method name="ReadRecord" />
    <method name="GetValue">
      <parameter name="intFieldIndex"/>
    </method>
    <method name="CloseInput">
      <parameter name="blnAbort"/>
    </method>
  </implements>
  <SCRIPT LANGUAGE="VBScript">

Option Explicit

Dim objAdminManager,objSessionDictionary
Dim objSitesSection,objSitesCollection
Dim objSiteElement,objFtpServerElement
Dim objSessionsElement,objSessionElement
Dim intSiteElementPos,intSession,intRecordIndex
Dim clsSession

intRecordIndex = -1

' --------------------------------------------------------------------------------
' Open an input session that reads FTP RSCA data and stores it in a dictionary object.
' --------------------------------------------------------------------------------

Public Function OpenInput(strValue)
  Set objSessionDictionary = CreateObject("Scripting.Dictionary")
  Set objAdminManager = CreateObject("Microsoft.ApplicationHost.WritableAdminManager")
  objAdminManager.CommitPath = "MACHINE/WEBROOT/APPHOST"
  Set objSitesSection = objAdminManager.GetAdminSection("system.applicationHost/sites", "MACHINE/WEBROOT/APPHOST")
  Set objSitesCollection = objSitesSection.Collection
  If IsNumeric(strValue) Then
    intSiteElementPos = FindElement(objSitesCollection, "site", Array("id", strValue))
  Else
    intSiteElementPos = FindElement(objSitesCollection, "site", Array("name", strValue))
  End If
  If intSiteElementPos > -1 Then
    Set objSiteElement = objSitesCollection.Item(intSiteElementPos)
    Set objFtpServerElement = objSiteElement.ChildElements.Item("ftpServer")
    Set objSessionsElement = objFtpServerElement.ChildElements.Item("sessions").Collection
    For intSession = 0 To CLng(objSessionsElement.Count)-1
       Set objSessionElement = objSessionsElement.Item(intSession)
       Set clsSession = New Session
       clsSession.CurrentDateTime = GetUtcDate()
       clsSession.ClientIp = objSessionElement.GetPropertyByName("clientIp").Value
       clsSession.SessionId = objSessionElement.GetPropertyByName("sessionId").Value
       clsSession.SessionStartTime = objSessionElement.GetPropertyByName("sessionStartTime").Value
       clsSession.UserName = objSessionElement.GetPropertyByName("userName").Value
       clsSession.CurrentCommand = objSessionElement.GetPropertyByName("currentCommand").Value
       clsSession.PreviousCommand = objSessionElement.GetPropertyByName("previousCommand").Value
       clsSession.CommandStartTime = objSessionElement.GetPropertyByName("commandStartTime").Value
       clsSession.BytesSent = objSessionElement.GetPropertyByName("bytesSent").Value
       clsSession.BytesReceived = objSessionElement.GetPropertyByName("bytesReceived").Value
       clsSession.LastErrorStatus = objSessionElement.GetPropertyByName("lastErrorStatus").Value
       objSessionDictionary.Add intSession,clsSession
    Next
  End If
End Function

' --------------------------------------------------------------------------------
' Close the input session.
' --------------------------------------------------------------------------------

Public Function CloseInput(blnAbort)
  intRecordIndex = -1
  objSessionDictionary.RemoveAll
End Function

' --------------------------------------------------------------------------------
' Return the count of fields.
' --------------------------------------------------------------------------------

Public Function GetFieldCount()
  GetFieldCount = 11
End Function

' --------------------------------------------------------------------------------
' Return the specified field's name.
' --------------------------------------------------------------------------------

Public Function GetFieldName(intFieldIndex)
  Select Case intFieldIndex
    Case 0
      GetFieldName = "currentDateTime"
    Case 1
      GetFieldName = "clientIp"
    Case 2
      GetFieldName = "sessionId"
    Case 3
      GetFieldName = "sessionStartTime"
    Case 4
      GetFieldName = "userName"
    Case 5
      GetFieldName = "currentCommand"
    Case 6
      GetFieldName = "previousCommand"
    Case 7
      GetFieldName = "commandStartTime"
    Case 8
      GetFieldName = "bytesSent"
    Case 9
      GetFieldName = "bytesReceived"
    Case 10
      GetFieldName = "lastErrorStatus"
  End Select
End Function

' --------------------------------------------------------------------------------
' Return the specified field's type.
' --------------------------------------------------------------------------------

Public Function GetFieldType(intFieldIndex)
  Const TYPE_INTEGER   = 1
  Const TYPE_REAL      = 2
  Const TYPE_STRING    = 3
  Const TYPE_TIMESTAMP = 4
  Const TYPE_NULL      = 5
  
  Select Case intFieldIndex
    Case 0
      GetFieldType = TYPE_STRING
    Case 1
      GetFieldType = TYPE_STRING
    Case 2
      GetFieldType = TYPE_STRING
    Case 3
      GetFieldType = TYPE_STRING
    Case 4
      GetFieldType = TYPE_STRING
    Case 5
      GetFieldType = TYPE_STRING
    Case 6
      GetFieldType = TYPE_STRING
    Case 7
      GetFieldType = TYPE_STRING
    Case 8
      GetFieldType = TYPE_INTEGER
    Case 9
      GetFieldType = TYPE_INTEGER
    Case 10
      GetFieldType = TYPE_INTEGER
  End Select
End Function

' --------------------------------------------------------------------------------
' Return the specified field's value.
' --------------------------------------------------------------------------------

Public Function GetValue(intFieldIndex)
  If objSessionDictionary.Count > 0 Then
    Select Case intFieldIndex
      Case 0
        GetValue = objSessionDictionary(intRecordIndex).CurrentDateTime
      Case 1
        GetValue = objSessionDictionary(intRecordIndex).ClientIp
      Case 2
        GetValue = objSessionDictionary(intRecordIndex).SessionId
      Case 3
        GetValue = objSessionDictionary(intRecordIndex).SessionStartTime
      Case 4
        GetValue = objSessionDictionary(intRecordIndex).UserName
      Case 5
        GetValue = objSessionDictionary(intRecordIndex).CurrentCommand
      Case 6
        GetValue = objSessionDictionary(intRecordIndex).PreviousCommand
      Case 7
        GetValue = objSessionDictionary(intRecordIndex).CommandStartTime
      Case 8
        GetValue = objSessionDictionary(intRecordIndex).BytesSent
      Case 9
        GetValue = objSessionDictionary(intRecordIndex).BytesReceived
      Case 10
        GetValue = objSessionDictionary(intRecordIndex).LastErrorStatus
    End Select
  End If
End Function
  
' --------------------------------------------------------------------------------
' Read the next record, and return true or false if there is more data.
' --------------------------------------------------------------------------------

Public Function ReadRecord()
  If objSessionDictionary.Count > 0 Then
    If intRecordIndex < (objSessionDictionary.Count-1) Then
      intRecordIndex = intRecordIndex + 1
      ReadRecord = True
    Else
      ReadRecord = False
    End If
  End If
End Function

' --------------------------------------------------------------------------------
' Return the current UTC date/time.
' --------------------------------------------------------------------------------

Private Function GetUtcDate()
  Dim dtmNow,dtmUtc,strUtc
  Dim objShell,lngActiveTimeBias
  dtmNow = Now()
  Set objShell = CreateObject("WScript.Shell")
  lngActiveTimeBias = CLng(objShell.RegRead("HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\TimeZoneInformation\ActiveTimeBias"))
  dtmUtc = DateAdd("n",lngActiveTimeBias,dtmNow)
  strUtc = Year(dtmUtc) & "-" & _
    Right("0" & Month(dtmUtc),2) & "-" & _
    Right("0" & Day(dtmUtc),2) & "T" & _
    Right("0" & Hour(dtmUtc),2) & ":" & _
    Right("0" & Minute(dtmUtc),2) & ":" & _
    Right("0" & Second(dtmUtc),2) & ".000Z"
  GetUtcDate = strUtc
End Function

' --------------------------------------------------------------------------------
' Return an element's position in a collection.
' --------------------------------------------------------------------------------

Private Function FindElement(objCollection, strElementTagName, arrValuesToMatch)
  Dim i,elem,matches,j,prop,value
  For i = 0 To CInt(objCollection.Count) - 1
    Set elem = objCollection.Item(i)
    If elem.Name = strElementTagName Then
      matches = True
      For j = 0 To UBound(arrValuesToMatch) Step 2
        Set prop = elem.GetPropertyByName(arrValuesToMatch(j))
        value = prop.Value
        If Not IsNull(value) Then
          value = CStr(value)
        End If
        If Not value = CStr(arrValuesToMatch(j + 1)) Then
          matches = False
          Exit For
        End If
      Next
      If matches Then
        Exit For
      End If
    End If
  Next
  If matches Then
    FindElement = i
  Else
    FindElement = -1
  End If
End Function

' --------------------------------------------------------------------------------
' Define a generic class for holding session data.
' --------------------------------------------------------------------------------

Class Session
  Public CurrentDateTime
  Public ClientIp
  Public SessionId
  Public SessionStartTime
  Public UserName
  Public CurrentCommand
  Public PreviousCommand
  Public CommandStartTime
  Public BytesSent
  Public BytesReceived
  Public LastErrorStatus
End Class

  </SCRIPT>

</SCRIPTLET>

After you've saved the scriptlet code to your computer, you will register it by using the following syntax:

regsvr32 MSUtil.LogQuery.FtpRscaScriptlet.sct

At the very minimum, you can now use the COM plug-in with Log Parser by using syntax like the following:

logparser "SELECT * FROM ftp.example.com" -i:COM -iProgID:MSUtil.LogQuery.FtpRscaScriptlet

Next, we'll analyze what the scriptlet does.

Examining the COM plug-in in Detail

Here are the different parts of the scriptlet and what they do:

  • The <registration> section of the scriptlet sets up the COM registration information; you'll notice the COM component class name and GUID, as well as version information and a general description.
  • The <implements> section declares the public methods that the COM plug-in has to support.
  • The <script> section contains the actual implementation:
    • The first part of the script section declares the global variables that will be used.
    • The second part of the script contains the required methods:
      • The OpenInput() method opens the FTP RSCA data for a specific FTP site:
        • The site will be specified in your Log Parser query, and the COM plug-in supports using either the site name or the site's numeric ID:
          • "SELECT * FROM ftp.example.com"
          • "SELECT * FROM 1"
        • All of the RSCA data for the site in stored in classes that will be cached in a global dictionary for quick access
      • The  CloseInput() method doesn't do much in this script, but your COM plug-ins may require more clean up depending on your data source.
      • The GetFieldCount() method simply returns the number of data fields in each record of your data.
      • The GetFieldName() method returns the name of a field that is passed to the method as a number.
      • The GetFieldType() method returns the data type of a field that is passed to the method as a number; Log Parser supports the following five data types for COM plug-ins:
        • TYPE_INTEGER
        • TYPE_REAL
        • TYPE_STRING
        • TYPE_TIMESTAMP
        • TYPE_NULL
      • The GetValue() method returns the data value of a field that is passed to the method as a number.
      • The ReadRecord() method moves to the next record in your data set; this method returns True if there is data to read, or False when the end of data is reached.
    • The third part of the script contains some helper features:
      • The GetUtcDate() method returns the current date and time in Universal Coordinated Time (UTC) format.
      • The FindElement() method locates a specified element's position within an IIS collection, or -1 if the element cannot be found. This method is used to determine the specified FTP site within the IIS configuration.
      • The Session class is a generic construct to hold the information for a single FTP RSCA data record.

This wraps up the description of how the scriptlet works as a COM plug-in, in the next part of my blog we'll look at how to actually use it.

Using the COM plug-in with Log Parser

Earlier I showed you how you can use the COM plug-in with Log Parser by using syntax like the following:

logparser "SELECT * FROM ftp.example.com" -i:COM -iProgID:MSUtil.LogQuery.FtpRscaScriptlet

This will return output that resembles something like the following:

currentDateTime clientIp sessionId sessionStartTime userName currentCommand previousCommand commandStartTime bytesSent bytesReceived lastErrorStatus
---------------- -------- --------- ---------------- -------- -------------- --------------- ---------------- --------- ------------- ---------------
2012-05-25T11:42:11.000Z 10.121.75.26 3950d1e5-3e94-4734-a89a-9768c52aa924 2012-05-25T10:08:09.861Z robert PASS USER 2012-05-25T11:42:06.080Z 6049 1193 0
2012-05-25T11:42:11.000Z 10.121.75.26 d1591fa8-3b09-4afd-b2c0-950421ba79fe 2012-05-25T10:08:18.184Z robert RETR NLST 2012-05-25T11:42:07.172Z 5887 1169 0
2012-05-25T11:42:11.000Z 10.121.75.26 0f92b5ed-920a-441d-a15d-39056a36f2a4 2012-05-25T10:08:22.327Z robert NOOP NLST 2012-05-25T11:41:40.917Z 5857 1163 0
2012-05-25T11:42:11.000Z 10.121.75.26 16925f0d-1fc5-4cb7-be19-ab33face2da9 2012-05-25T10:08:48.756Z   NLST SYST 2012-05-25T11:41:44.770Z 6026 1192 0
2012-05-25T11:42:11.000Z 10.121.75.26 aeb68389-869b-4afc-8c81-47b578e74824 2012-05-25T10:08:54.214Z   USER HOST 2012-05-25T11:41:42.087Z 5864 1168 0
2012-05-25T11:42:11.000Z 10.121.75.26 4ed55569-ee25-47d1-8388-12cdb90a1c07 2012-05-25T10:12:31.555Z alice RETR NLST 2012-05-25T11:42:01.789Z 5780 1138 0
2012-05-25T11:42:11.000Z 10.121.75.26 d6b16bb4-cb65-492d-a9fa-fbd6b72de0f3 2012-05-25T10:12:54.591Z bob NOOP NLST 2012-05-25T11:41:46.563Z 5748 1130 0
                     
Statistics:                    
-----------                    
Elements processed: 7                  
Elements output: 7                  
Execution time: 0.12 seconds                  

That information is something of a jumbled mess, and we can clean that up a bit by simply choosing the fields that we might be interested in:

userName currentCommand commandStartTime
-------- -------------- ----------------
robert PASS 2012-05-25T11:42:06.080Z
robert RETR 2012-05-25T11:42:07.172Z
robert NOOP 2012-05-25T11:41:40.917Z
  NLST 2012-05-25T11:41:44.770Z
  USER 2012-05-25T11:41:42.087Z
alice RETR 2012-05-25T11:42:01.789Z
bob NOOP 2012-05-25T11:41:46.563Z
     
Statistics:    
-----------    
Elements processed: 7  
Elements output: 7  
Execution time: 0.12 seconds  

Now let's look at some interesting data - one of the main focuses for this blog series is charting  with Log Parser, so let's look at doing something useful with the data. To start with, here's how to create a pie chart that counts the number of sessions by user name:

logparser "SELECT CASE UserName WHEN '' THEN 'anonymous' ELSE TO_LOWERCASE(UserName) END AS User,COUNT(*) AS Sessions INTO SessionCountByUser.gif FROM 1 GROUP BY User" -i:COM -iProgID:MSUtil.LogQuery.FtpRscaScriptlet -chartType:PieExploded -o:CHART -values:on -categories:off -legend:on -chartTitle:"User Sessions"

This will generate a chart like the following:

Here's a variation on that script that illustrates how to create a pie chart that counts the number of authenticated sessions versus anonymous sessions:

logparser "SELECT CASE UserName WHEN '' THEN 'Anonymous' ELSE 'Authenticated' END AS AuthType,COUNT(*) AS Sessions INTO AuthenticatedSessions.gif FROM 1 GROUP BY AuthType" -i:COM -iProgID:MSUtil.LogQuery.FtpRscaScriptlet -chartType:PieExploded -o:CHART -values:on -categories:off -legend:on -chartTitle:"Authenticated Sessions"

This will generate a chart like the following:

We can also do line, bar, and column charts with the data:

logparser "SELECT CASE UserName WHEN '' THEN 'anonymous' ELSE TO_LOWERCASE(UserName) END AS User,COUNT(*) AS Sessions INTO SessionCountByUser.gif FROM 1 GROUP BY User" -i:COM -iProgID:MSUtil.LogQuery.FtpRscaScriptlet -chartType:Column3D -o:CHART -values:on -legend:off -chartTitle:"User Sessions"

The above code sample will generate a chart like the following:

There's a lot more that we could do with this, but eventually I have to get some sleep, so I think that's enough fun for the day.

Summary

In this blog post, I've shown you how to add your own custom input format to Log Parser by creating scriptlet as a COM plug-in. I hope that you take this information and create some great Log Parser plug-ins of your own.

;-]

Posted: May 24 2012, 22:47 by Bob | Comments (0)
  • Currently 0/5 Stars.
  • 1
  • 2
  • 3
  • 4
  • 5
Filed under: IIS | LogParser
Social Bookmarks: E-mail | Kick it! | DZone it! | del.icio.us

Advanced Log Parser Charts Part 3 - Missing Office Web Components for Charting

In Part 3 of this series, I'll explain what to do when you're missing the Office Web Components that are required for creating the charts that I have been demonstrating in this series.

Here's a brief explanation of the symptoms: you try a simple query that will create a chart like the following example:

logparser.exe "SELECT Date,COUNT(*) AS Hits INTO HitsByDay.gif FROM *.log GROUP BY Date ORDER BY Date" -i:W3C -o:CHART -chartType:Line -legend:off -chartTitle:"Hits By Day"

And you get the following error message:

Error creating output format "CHART": This output format requires a licensed Microsoft Office Chart Web Component to be installed on the local machine

More often than not, this simply means that you have Office 2007 or Office 2010, which do not contain the Office Web Components that are used by Log Parser to create charts. Fortunately, you can download the missing components from the following URL on Microsoft's website:

Office 2003 Add-in: Office Web Components
http://www.microsoft.com/en-us/download/details.aspx?id=22276

When you run the installation, you will see the following license agreement:

When you check the box to accept the license agreement and click Install, you will eventually receive the following dialog box to let you know that the Office 2003 Web Components have been installed:

Once you have the Office 2003 Web Components installed, you can run the same query successfully:

logparser.exe "SELECT Date,COUNT(*) AS Hits INTO HitsByDay.gif FROM *.log GROUP BY Date ORDER BY Date" -i:W3C -o:CHART -chartType:Line -legend:off -chartTitle:"Hits By Day"

Statistics:
-----------
Elements processed: 123330
Elements output: 14
Execution time: 0.57 seconds

Note: The above query generates the following somewhat uninteresting chart:

That being said, the point of this blog was to let you know how to get charting back, not how to make pretty charts. I'll save pretty charts for a future blog. ;-]

Note: This blog was originally posted at http://blogs.msdn.com/robert_mcmurray/
Posted: May 24 2012, 19:40 by Bob | Comments (0)
  • Currently 0/5 Stars.
  • 1
  • 2
  • 3
  • 4
  • 5
Filed under: IIS | LogParser
Social Bookmarks: E-mail | Kick it! | DZone it! | del.icio.us