Datasets
Important
This topic and its sub-topics provides the JSON format that is supported by the older versions of Azure PowerShell. If you are using the July 2015 Release version of Azure PowerShell or later, see Datasets for the latest JSON format. You can convert the JSON from old format to new format by using the tool: JSON Upgrade Tool
A dataset is a named view of data. The data being described can vary from simple bytes, semi-structured data like CSV files all the way up to relational tables or even models. A Data Factory table is a data set that has a schema and is rectangular.
The top-level structure for the Table JSON is as follows:
{
"name": "<name of table>",
"properties":
{
"structure": [ ],
"location":
{
"type": "<type of table>",
"linkedServiceName": "MyLinkedService"
},
"availability":
{
},
"policy":
{
}
}
}
The following table describes top-level properties in the JSON script for a table.
Tag |
Description |
Required |
Name |
Name of the table. |
Yes |
structure |
Contains an array of columns (Name and Type). See Structure Element topic for details. |
No The structure section is only required when using column mapping It is optional in other cases. |
location |
Defines the format of the data and where to find the data. See Location Element topic for the types of tables supported and their location properties. |
Yes |
Availability |
Defines the cadence (hourly, daily, weekly, etc…) in which a slice of the table is produced. If the data is produced outside of the data factory, you should use waitOnExternal property and specify the duration for waiting for the external data to be available. See Availability Element topic for more details. |
Yes |
Policy |
Specifies the policies for each run of the pipelines that produces the table. See the Policy Element topic for details. |
No |