System.ApplicationLog.GenericLogReader

System Center

Updated: August 10, 2011

Applies To: Operations Manager 2007 R2

The System.ApplicationLog.GenericLogReader data source module type is used to provide log entry data from a directory that contains non-delimited text-based generic log files. This module returns each generic log file’s entry data as System.ApplicationLog.GenericLogEntryData data.

Usage

Use the module to provide generic log entry data from within a workflow. A standard generic log file is a file that contains only one non-delimited string per log entry. If your log file includes delimited text within each entry, use the System.ApplicationLog.GenericLogReader module type.

Type Definition


<DataSourceModuleType ID="System.ApplicationLog.GenericLogReader" Accessibility="Public">
  <Configuration>
    <xsd:element name="LogFileDirectory" type="xsd:string" />
    <xsd:element name="LogFilePattern" type="xsd:string" />
    <xsd:element name="LogIsUTF8" type="xsd:string" />
  </Configuration>
  <ModuleImplementation>
    <Composite>
      <MemberModules>
        <DataSource TypeID="System.ApplicationLog.GenericLogReader.Internal" ID="DS">
          <LogFileType>6</LogFileType>
          <LogFileDirectory>$Config/LogFileDirectory$</LogFileDirectory>
          <LogFilePattern>$Config/LogFilePattern$</LogFilePattern>
          <LogIsUTF8>$Config/LogIsUTF8$</LogIsUTF8>
        </DataSource>
      </MemberModules>
      <Composition>
        <Node ID="DS" />
      </Composition>
    </Composite>
  </ModuleImplementation>
  <OutputType>System.ApplicationLog.GenericLogEntryData</OutputType>
</DataSourceModuleType>

Parameters

The System.ApplicationLog.GenericLogReader module supports the configuration parameters described in the following table.

 

Parameter Type Overrideable Description

LogFileDirectory

String

False

Required parameter. Contains the full path to the local directory to read the log file or files from.

LogFilePattern

String

False

Required parameter. Contains the file name or wildcard pattern to match for the log files.

LogIsUTF8

Boolean

False

Required parameter. Indicates whether the log file is UTF8-encoded. The value should be either true or false.

LogFileDirectory

The LogFileDirectory parameter specifies an absolute path to the log file or files or to a path by using environment variables or $Target variables or both. Wildcards cannot be used in this parameter, because a single workflow can look for files only in a single directory by using this module type. Absolute paths are generally not recommended, because many applications can be installed to non-default directories by the user.

The following code shows an example of an absolute file path:

<LogFileDirectory>C:\ApplicationX</LogFileDirectory>

The following code shows the use of an environment variable:

<LogFileDirectory>%ProgramFiles%\ApplicationX</LogFileDirectory>

The following code shows the use of a discovered property of an object:

<LogFileDirectory>$Target/Property[Type='Microsoft.Samples.ApplicationX']/LogFilePath$</LogFileDirectory>

LogFilePattern

The LogFilePattern parameter can contain an absolute file name reference, as shown in the following example:

<LogFilePattern>AppX.log</LogFilePattern> 

In the preceding example, only a single log file is ever used as the source for the module. The module maintains a bookmark to the current position of the log file and acts only on new log file entries.

The parameter can also contain a wildcard reference to file names, as shown in the following example:

<LogFilePattern>AppX*.log</LogFilePattern> 

This parameter is designed to work with applications that create new log files when the current log file gets to a certain size or when a new file is created based on the advancement of time, such as a new file that is created every day.

ImportantImportant
This module maintains a bookmark only to the most recent file and is designed to work with only a single file at a time. If there are multiple files being written to in alternating order, the module does not output data as expected.

Remarks

On the first initiation of the workflow, the module attempts to read the specified directory. The module then looks for the most recent file that matches the file pattern and reads next entry.

When a newer file that matches the file pattern is created, the module switches to gather events from the newer file. At any time, the module holds a bookmark to only a single file.

The module does not support log-file wrapping.

WarningWarning
When reading a log file, the last line read is stored so that the module does not process all lines on every read. If all lines of the current file are deleted and writing begins at the first line again, the module will not read the new line. To work around this, you must make sure a new file is created instead of overwriting the old file.

Composition

The System.ApplicationLog.GenericLogReader module is a composite module that contains the member modules described in the following table.

 

Workflow Run Order Module Type Usage

1

System.ApplicationLog.GenericLogReader.Internal

Internal module.

Errors

The following errors can occur from within the System.ApplicationLog.GenericLogReader module.

 

EventID Reason

31708

The log directory specified in the LogFileDirectory parameter can no longer be found.

31709

The log directory specified in the LogFileDirectory parameter cannot be accessed.

31710

The current log file being processed has suddenly disappeared from the directory.

31711

The current log file being processed cannot be accessed. The module will continue to try and read the file.

Related Modules

 

Module Type Usage

System.ApplicationLog.GenericCSVLog.EventProvider

Maps delimited, text-based generic log entry data to event data. Returns System.Event.Data data.

System.ApplicationLog.GenericCSVLogReader

Maps filtered delimited, text-based generic log entry data to event data. Returns System.Event.Data data.

External Module References

The System.ApplicationLog.GenericLogReader module is a member of the modules described in the following table.

 

Module Type Library Usage

System.ApplicationLog.GenericLog.EventProvider

System.ApplicationLog.Library

Maps non-delimited, text-based generic log entry data to event data. Returns System.Event.Data data.

System.ApplicationLog.GenericLog.FilteredEventProvider

System.ApplicationLog.Library

Maps filtered and non-delimited, text-based generic log entry data to event data. Returns System.Event.Data data.

Sample

The following sample illustrates how a rule can be implemented to retrieve generic log entry data from any number of log files in a directory. In this sample, the generic log entry data is mapped to System.Event.Data so that it can be stored in the Operations Manager database. This sample is for illustrative purposes only. Generally, to write log entry data as event data, you would use either the System.ApplicationLog.GenericLog.EventProvider module type or the System.ApplicationLog.GenericLog.FilteredEventProvider module type.


<Rule ID="Microsoft.Samples.LogEntryToEvent" Target="Microsoft.Samples.TheApplication" Enabled="true">
 <DataSources>
    <DataSource ID="LogDS" TypeID="AppLog!System.ApplicationLog.GenericLogReader">
      <LogFileDirectory>C:\logs</LogFileDirectory>
      <LogFilePattern>*.log</LogFilePattern>
      <LogIsUTF8>false</LogIsUTF8>
    </DataSource>
  </DataSources>
  <ConditionDetection TypeID="System!System.Event.GenericDataMapper" ID="Mapper">
    <EventOriginId>$Target/Id$</EventOriginId>
    <PublisherId>$MPElement$</PublisherId>
    <PublisherName>GenericLog</PublisherName>
    <Channel>GenericLog</Channel>
    <LoggingComputer />
    <EventNumber>0</EventNumber>
    <EventCategory>3</EventCategory>
    <EventLevel>0</EventLevel>
    <UserName />
    <Params>
      <Param>$Data/Params/Param[1]$</Param>
    </Params>
  </ConditionDetection>
  <WriteActions>
    <WriteAction ID="WriteToDB" TypeID="SCLibrary!Microsoft.SystemCenter.CollectEvent"/>
    <WriteAction ID="WriteToDW" TypeID="SCDW!Microsoft.SystemCenter.DataWarehouse.PublishEventData"/>
  </WriteActions>

</Rule>

In the preceding sample, the value of the Param element in the mapper module is $Data/Params/Param[1]$, which contains the text of the log entry. The $Data variable notation refers to the data structure that is returned from the System.ApplicationLog.GenericLogReader module. If the log entry line is This is one log entry, the data structure would be something like the following:


<DataItem type="System.ApplicationLog.GenericLogEntryData" time="2008-03-11T03:08:55.0343534-08:00" sourceHealthServiceId="0A0800A0-A802-E90B-6045-D961D516CA78">
  <LogFileDirectory>C:\logs</LogFileDirectory>
  <LogFileType>6</LogFileType>
  <LogFileName>C:\logs\somelog.log</LogFileName>
  <Params>
     <Param>This is one log entry.</Param>
  </Params>
</DataItem>

The data item from the generic event mapper would look like the following:


<DataItem type="System.Event.Data" time="2008-03-11T03:08:55.0343534-08:00" sourceHealthServiceId="0A0800A0-A802-E90B-6045-D961D516CA78">
  <EventOriginId>{513C3E6D-374A-47d1-94B6-920EB54F9A27}</EventOriginId>
  <PublisherId>{513C3E6D-374A-47d1-94B6-920EB54F9A27}</PublisherId>
  <PublisherName>GenericLog</PublisherName>
  <Channel>GenericLog</Channel>
  <LoggingComputer />
  <EventNumber>0</EventNumber>
  <EventCategory>3</EventCategory>
  <EventLevel>0</EventLevel>
  <UserName />
  <EventData>
    <DataItem type="System.ApplicationLog.GenericLogEntryData" time="2007-05-22T10:07:41.6416394-07:00" sourceHealthServiceId="A9BB62D3-BEF2-5208-E680-1EE489235408">
      <LogFileDirectory>c:\Logs</LogFileDirectory>
      <LogFileType>Generic Log File Format</LogFileType>
      <LogFileName>c:\Logs\some.log</LogFileName>
      <Params>
         <Param>This is one log entry.</Param>
      </Params>
      <DataItem>
  </EventData>
</DataItem>


Information

 

   

Module Type

DataSourceModuleType

Input Type

None

Output Type

System.ApplicationLog.GenericLogEntryData

Implementation

Composite

Library

System.ApplicationLog.Library

 
Show: