Define and Format Data

Define and Format Data

In Step 1 of the Data Import Routine, you will identify how the data is formatted and define where the data values start and stop.

 

Define and format data

  1. Step 1 of the Data Import Routine appears as follows:
  3. (Optional) If you have already saved an Import Template for this type of file and want to use it, click Load Template The Load Import Template window will appear (See Import Templates for more details).

Select the appropriate template and click OK. This will load the parameters (including the mapping decisions) that have been saved for that import specification.

  1. If you have not loaded an Import Template, then set the appropriate Field Delimiter, Text Qualifier, Date Format, Date Delimiter, Time Delimiter and the Decimal Symbol.
  2. (Optional) Check Normalised if your data is in a normalised format. Otherwise leave it unchecked (See Data Import for more details).
  3. (Optional) Check Midnight is End of Day if you wish to write any raw data records showing timestamps of 24:00 to 23:59:59. Ticking the checkbox for Midnight is End of Day on import ensures that data reading as 24 hours in the raw data file is recorded in DataSight as 23:59:59 and is grouped within the nominated date and hourly filters. When the box is not checked, a 24:00 hour reading will write to the DataSight database as 00:00:00 or 12:00:00 AM, and subsequently be grouped within the next day of data.
  4. (Optional) Check Enable Time Offset to automatically increment the datetime stamp by five seconds when replicate records are written to the DataSight database. This feature may be particularly important when importing replicate laboratory analyses, which have the same level, timestamp and variable. The subtle change in datetime ensures both replicates for the same variable are preserved. This feature is only available during import and for data found within the confines of one singular data file.
  5. Use the scroll arrows to set the First Title Row and Number of Title Rows will appear next to the first and last Title Row lines in the data panel.
  6. Scroll down the data panel and identify where the data starts. Position the cursor on the appropriate line number in the data panel and left-click. This will set the position of the First Data Row and will appear next to that line in the data panel. Any values above this row (apart from any set Title rows) will be ignored in the Import process. Continue to scroll down the data panel and identify where the data stops. Right-click on the appropriate line number in the data panel to set the position of the Last Data Row and will appear next to that line in the data panel. Any values below this line will be ignored in the Import process.
  7. Click Next. The Step 2 Data Import Routine window appears.

Note
  • When importing a worksheet from a workbook you must import and parse each worksheet separately. Use the Sheet drop-down menu to select any worksheet.
  • When importing data, if the first line in the file is data and not a column header leave “First Title Row” as “0”.
  • Date and Time Delimiters must be precisely identified. Data validation is strictly enforced and any change in date or time formatting from record to record within a raw data file will result in the partial transfer of your data to the database.

 

See also:

  • Step 2: Filter and Map Your Data

    • Related Articles

    • Data Import

      Data Import Checklist Before importing data, you should be able to answer the following questions about the current format of your file. What type of file are you importing? Is the data normalised or pivoted? Do the data variables you wish to import ...

    • Import Data

      Open a desired Station Table. Click Import Data. Select Manual Data Entry or Import Data File. Manual Data Entry Enter date, time and your variable value. Click Save. Import Data File Browse to your file and select Open, then click Upload. Configure ...

    • Data Migration

      Data migration is the process of transferring data between storage types, formats, or computer systems. When is data migration performed Every time you import newly acquired data into DataSight you are migrating data from one format to another. The ...

    • Preview Data

      In Step 3 of the Data Import Routine, preview the data you are about to import. Click Back to step backwards through the Import Routine to change any formatting, filtering or mapping as required. Click Next if you are satisfied with the data set. See ...

    • Data List

      A Data List navigation tree can be accessed on the left side of your DSWeb page, to view data and information relating to a Level 3 station or site. Monitoring variables are available for viewing in both tabular and graphical formats. View Data List ...