Export (0) Print
Expand All

New-AzureHDInsightMapReduceJobDefinition

[This topic is pre-release documentation and is subject to change in future releases. Blank topics are included as placeholders.]

New-AzureHDInsightMapReduceJobDefinition

Defines a new Map Reduce job.

Syntax

Parameter Set: Default
New-AzureHDInsightMapReduceJobDefinition -ClassName <String> -JarFile <String> [-Arguments <String[]> ] [-Defines <Hashtable> ] [-Files <String[]> ] [-JobName <String> ] [-LibJars <String[]> ] [-StatusFolder <String> ] [ <CommonParameters>]




Detailed Description

Defines a new Map Reduce job to be run on an Azure HDInsight cluster.

This topic describes the cmdlet in the 0.8.1 version of the Microsoft Azure PowerShell module. To find out the version of the module you're using, from the Azure PowerShell console, type (get-module azure).version.

Parameters

-Arguments<String[]>

Arguments of the Hadoop job. The arguments will be passed as command line arguments to each task.


Aliases

Args

Required?

false

Position?

named

Default Value

none

Accept Pipeline Input?

false

Accept Wildcard Characters?

false

-ClassName<String>

Name of the job class in the job JAR file.


Aliases

Class

Required?

true

Position?

named

Default Value

none

Accept Pipeline Input?

false

Accept Wildcard Characters?

false

-Defines<Hashtable>

Hadoop configuration values to be set during the job execution.


Aliases

Params

Required?

false

Position?

named

Default Value

none

Accept Pipeline Input?

false

Accept Wildcard Characters?

false

-Files<String[]>

The collection of files required for the execution of the job. Use wasb file references here.


Aliases

none

Required?

false

Position?

named

Default Value

none

Accept Pipeline Input?

false

Accept Wildcard Characters?

false

-JarFile<String>

The fully qualified name of the JAR file that contains the code and dependencies of the MapReduce job.


Aliases

Jar

Required?

true

Position?

named

Default Value

none

Accept Pipeline Input?

false

Accept Wildcard Characters?

false

-JobName<String>

The name of the MapReduce job. Specifying the name is optional. If it is not explicitly specified the value of the ClassName parameter is used by default.


Aliases

Name

Required?

false

Position?

named

Default Value

none

Accept Pipeline Input?

false

Accept Wildcard Characters?

false

-LibJars<String[]>

LibJar references of the job.


Aliases

none

Required?

false

Position?

named

Default Value

none

Accept Pipeline Input?

false

Accept Wildcard Characters?

false

-StatusFolder<String>

Aliases

none

Required?

false

Position?

named

Default Value

none

Accept Pipeline Input?

false

Accept Wildcard Characters?

false

<CommonParameters>

This cmdlet supports the common parameters: -Verbose, -Debug, -ErrorAction, -ErrorVariable, -OutBuffer, and -OutVariable. For more information, see  about_CommonParameters (http://go.microsoft.com/fwlink/p/?LinkID=113216).

Inputs

The input type is the type of the objects that you can pipe to the cmdlet.

Outputs

The output type is the type of the objects that the cmdlet emits.

Examples

 

 

Define a Map Reduce job.


 

Define a new Map Reduce job to be run on a specified Azure HDInsight cluster..


PS C:\> $subid = (Get-AzureSubscription –Current).SubscriptionId$clustername = “yourcluster”$wordCountJob = New-AzureHDInsightMapReduceJobDefinition -JarFile "/example/apps/hadoop-examples.jar" -ClassName "wordcount" `   -Defines @{ "mapred.map.tasks"="3" } -Arguments "/example/data/gutenberg/davinci.txt", "/example/output/WordCount"$wordCountJob `    | Start-AzureHDInsightJob -Cluster $clustername `    | Wait-AzureHDInsightJob -Subscription $subid -WaitTimeoutInSeconds 3600 `    | Get-AzureHDInsightJobOutput -Cluster $clustername -Subscription $subid -StandardError

Show:
© 2014 Microsoft