Was this page helpful?
Your feedback about this content is important. Let us know what you think.
Additional feedback?
1500 characters remaining
Export (0) Print
Expand All

New-AzureHDInsightStreamingMapReduceJobDefinition

[This topic is pre-release documentation and is subject to change in future releases. Blank topics are included as placeholders.]

New-AzureHDInsightStreamingMapReduceJobDefinition

Defines a new streaming Map Reduce job.

Syntax

Parameter Set: Default
New-AzureHDInsightStreamingMapReduceJobDefinition [-Arguments <String[]> ] [-CmdEnv <String[]> ] [-Combiner <String> ] [-Defines <Hashtable> ] [-Files <String[]> ] [-InputPath <String> ] [-JobName <String> ] [-Mapper <String> ] [-OutputPath <String> ] [-Reducer <String> ] [-StatusFolder <String> ] [ <CommonParameters>]




Detailed Description

Defines new job definition object that represents parameters of the Hadoop streaming job.

This topic describes the cmdlet in the 0.8.1 version of the Microsoft Azure PowerShell module. To find out the version of the module you're using, from the Azure PowerShell console, type (get-module azure).version.

Parameters

-Arguments<String[]>

Arguments of the Hadoop job. The arguments will be passed as command line arguments to each task.


Aliases

Args

Required?

false

Position?

named

Default Value

none

Accept Pipeline Input?

false

Accept Wildcard Characters?

false

-CmdEnv<String[]>

Command line environmental variables that should be set during job execution on data nodes.


Aliases

none

Required?

false

Position?

named

Default Value

none

Accept Pipeline Input?

false

Accept Wildcard Characters?

false

-Combiner<String>

Combiner executable name.


Aliases

none

Required?

false

Position?

named

Default Value

none

Accept Pipeline Input?

false

Accept Wildcard Characters?

false

-Defines<Hashtable>

Hadoop configuration values to be set during the job execution.


Aliases

Params

Required?

false

Position?

named

Default Value

none

Accept Pipeline Input?

false

Accept Wildcard Characters?

false

-Files<String[]>

The collection of files associated with the MapReduce job.


Aliases

none

Required?

false

Position?

named

Default Value

none

Accept Pipeline Input?

false

Accept Wildcard Characters?

false

-InputPath<String>

WASB path representing location of input files.


Aliases

Input

Required?

false

Position?

named

Default Value

none

Accept Pipeline Input?

false

Accept Wildcard Characters?

false

-JobName<String>

The name of the MapReduce job. Specifying the name is optional.


Aliases

Name

Required?

false

Position?

named

Default Value

none

Accept Pipeline Input?

false

Accept Wildcard Characters?

false

-Mapper<String>

Mapper executable name.


Aliases

none

Required?

false

Position?

named

Default Value

none

Accept Pipeline Input?

false

Accept Wildcard Characters?

false

-OutputPath<String>

WASB path representing location for the job output.


Aliases

Output

Required?

false

Position?

named

Default Value

none

Accept Pipeline Input?

false

Accept Wildcard Characters?

false

-Reducer<String>

Reducer executable name.


Aliases

none

Required?

false

Position?

named

Default Value

none

Accept Pipeline Input?

false

Accept Wildcard Characters?

false

-StatusFolder<String>

Contains the standard and error outputs for the job, including its exit code and task logs.


Aliases

none

Required?

false

Position?

named

Default Value

none

Accept Pipeline Input?

false

Accept Wildcard Characters?

false

<CommonParameters>

This cmdlet supports the common parameters: -Verbose, -Debug, -ErrorAction, -ErrorVariable, -OutBuffer, and -OutVariable. For more information, see  about_CommonParameters (http://go.microsoft.com/fwlink/p/?LinkID=113216).

Inputs

The input type is the type of the objects that you can pipe to the cmdlet.

Outputs

The output type is the type of the objects that the cmdlet emits.

Examples

 

 

Define a new streaming Map Reduce job.


 

Create a MapReduce job definition for the streaming job.


PS C:\> $subscriptionName = "<SubscriptionName>"   # Azure subscription name
$clusterName = "<ClusterName>"             # HDInsight cluster name

# Create a MapReduce job definition for the streaming job.
$streamingWC = New-AzureHDInsightStreamingMapReduceJobDefinition -Files "/example/apps/wc.exe", "/example/apps/cat.exe" -InputPath "/example/data/gutenberg/davinci.txt" -OutputPath "/example/data/StreamingOutput/wc.txt" -Mapper "cat.exe" -Reducer "wc.exe"

Show:
© 2015 Microsoft