This page will look better in a graphical browser that supports web standards, but is accessible to any browser or internet device.

Served by Samwise.

Cardiac Physiome Society workshop: November 6-9, 2017 , Toronto

Data Files and Project Data Sets

This page is for the current JSim version 2.0. Click here for the earlier JSim 1.6 version.

Introduction

JSim Data Sets are portions of a JSim project that represent numeric data captured from experimental measurements or previous model runs. This document describes how to load data sets into a JSim project, some typical uses for them once they are loaded, and how to export data to external files.

Prerequisites:

Contents:

JSim Data Set Structure

JSim data sets are collections of data curves. Each curve has a name that is unique within the data set by which the curve is known to the rest of JSim. A curve name must also start with a letter and contains only letters, numbers and the underscore character. A curve may optionally possess a free-form text description and physical unit information. Curves may be of any dimension, however 1-dimensional data is most common and is best supported in JSim at present. Each data set in a JSim project will appear as a leaf in the JSim project tree and as a tab on the left-hand side of the JSim project window.

Loading data sets from external files

Most data sets are loaded from external data files. Currently, the following import formats are supported:

JSim also offers a data format plug-in feature that allows users to define support for other file formats within JSim. Serious users of JSim should become familiar with the relative advantages of each supported format. The list of supported import formats will grow as JSim matures.

To load an external data file of a supported format:

1) obtain a copy of the file on your computer system and make sure if conforms to the applicable specification (see above);

2) select "Import data file ..." from the Add menu in a project's "Project" tab;

3) navigate the file system and select the desired file.

If the file conforms to specification, a new data set leaf will appear in the project tree. If not, a diagnostic error message will appear in the project's Message tab.

Loading data sets from model runs

It is sometimes useful to save output data from model runs into data sets. Some possible uses include:

To do this:

1) run a model and plot the desired data in a plot page;

2) select "Store project dataset ..." from the plot page's File menu;

3) name the new data set at the prompt.

Once completed a new data set leaf will appear in the project tree.

Note that a data set may be created from the model optimization graph in an analogous manner.

Copying data sets between projects

Data sets may be copied directly between project using the cut/copy/paste mechanism that applied to all items in a project tree. This is preferred to the old-style mechanism of writing data sets to external file and then reimporting the external files because:

To do this:

1) make sure both project windows were created in the same JSim program execution (copying between different executions is not yet supported);

2) select the Project tab in the project you wish to copy from;

3) highlight the data set (or other project tree leaf) you wish to copy by clicking on it. Additional leaves may be highlighted by clicking with the Control key depressed;

4) select "copy" from the Edit menu;

5) select the Project tab in the project you wish to copy to;

6) select "paste" from the Edit menu.

Typical uses for Data Sets

Plotting. Data may be selected for plotting in plot pages. Once a data set is loaded, it becomes an option in the data source selection box in the plot page configurator. This data source may be either a model or a data set. When it is a data set, the data select box will list the curves in a data set by name.

Function Generators. Model function generators are used to generate useful functions that can be assigned to model input variables. The "DataCurve" option allows the function generator to assume values taken from a particular curve in a data set. For further information see Using Function Generators in JSim

Optimization. JSim's model optimizers allow for automated analysis of what parameter values generate output that best match given experimental data. For further information see Using JSim's Parameter Optimizer

Exporting data sets to external files

Data files may currently be exported from two places within JSim:

Exported data files may be written in any of the supported input formats or in three "output only" formats below:

When exporting data, the user must supply a file suffix, and that file suffix must match the recommended suffix for the desired data format. Formats and corresponding suffixes are displayed in the export dialog. They are also detailed in this document under the appropriate format.

TAC Data Format

TAC (for Time Activity Curve) format is an ASCII text specification designed by NSR. It is strongly recommended that TAC files have the .tac extension. Complete TAC format specification may be found in the XSIM download section of the NSR website .

TAC format strengths are:

TAC format weaknesses are:

JSim assigns names to curves within TAC files using the same convention as XSIM. For those unfamiliar with XSIM, here is a brief synopsis. The highest level of TAC file structure is the "run". Each run may contain any number of physiological, input function and/or sample curves. JSim's assigned curve names are of the form R#x# where:

Columnar Text Data Format

Columnar text data files are ASCII files containing a single 2-dimensional table representing one or more 1-dimensional data curves. It is strongly recommended such file have the .cdata extension. The format is described below.

Columnar text format strengths:

Columnar text format weaknesses:

Comma Separated Value (CSV) Data Format

CSV data files are ASCII files containing a single 2-dimensional table representing one or more 1-dimensional data curves. They are very similar to Columnar text format except that they use the comma as a separator character. It is strongly recommended such file have the .csv extension.

CSV format strengths:

CSV format weaknesses:

Formal format specification:

File must be ASCII text. Blank lines are ignored. Each non-blank line must have the same number of columns. The first non-blank line may be either a header line or a data line. The second non-blank line may be either a units line or a data line. All subsequent non-blank lines must be data lines. Data lines must contain only space or tab delimited numbers. Numbers follow standard ASCII floating point representation. The keyword NaN, in any capitalization, translates to an IEEE 754 NaN. The first non-blank line is considered a header line if the content of first column is not a number. If present, a header line must contain a unique name for each column that starts with a letter. If present, a units line must contain a unit name for each column. Each name or unit may or may not be enclosed in matching single or double quotes. The first column represents the independent variable. Subsequent columns represent dependent variable values corresponding to the 1st column. Values of the independent variable must be strictly increasing and may be either regularly or irregularly spaced.

The imported data set will contain one curve for each dependent variable, that is, one less than the number of columns. If a header line is present, curve names are taken from the header with any enclosing quotes removed and any unacceptable characters replaced by underscore. If a header line is not present, curves are automatically named y1, y2, etc.

Examples:

Two examples should make the formal specification easier to grasp. The first file includes a header line. JSim will name the two resulting curves u and v. The columns line up in this example for clarity, but alignment is not required by the specification:

        time      u         v
        0         1         2
        1         .5        4
        2         .25       8
        5         .125      16
        10        6.25e-2   32

In this second, there is no header line. JSim will name the two resulting curves y1 and y2:

        0         1         2
        1         .5        4
        2         .25       8
        5         .125      16
        10        6.25e-2   32

In this third example, there is both a header an a units line:

        time      u         v
        sec       watt/M^2  m/sec
        0         1         2
        1         .5        4
        2         .25       8
        5         .125      16
        10        6.25e-2   32

JSML Data Format

This format (JSim Markup Language) is an XML dialect used for internal data storage within JSim project files. It supports multi-dimensional data and offers easy interchange with XML based data-bases and applications. XML tags for this format are not yet finalized. The recommended file suffix for this format is .jsml .

There are two encoding options for JSML: legacy and ascii. Ascii encoding is now recommended in all cases because:

Ascii is the default encoding for JSML data files. The jsbatch -oencoding switch allows you to specify legacy encoding if desired.

Pretty Data Format

This format presents 0-D, 1-D and 2-D in somewhat attractive text tables. Higher dimensional data is rather less attractive. This format is used for text display of data within JSim (e.g. plot page text). It is an output format only, and may not be imported into JSim. The recommended file suffix for this format is .pdata .

Line Data Format

This format presents data for each variable on a single line. This presents some advantages in model verification and debugging for researchers familiar with Unix text file tools. It is an output format only, and may not be imported into JSim. The recommended file suffix for this format is .ldata .

Matlab Data Format

This format allows export of JSim numeric results to Matlab via .m files. This is an output only format, and may not be imported into JSim. This format supports data of any dimensionality. A 1D MML variable will be exported as a Matlab vector, a 2D MML variable will be exported as a 2D Matlab matrix, a 3D MML variable will be exported as a 3D Matlab matrix, and so on. The domain (e.g. time) values corresponding the matrix dimensions are also written to Matlab vectors in a systematic way. For example, a 2D MML variable u(t,x) would generate 3 Matlab variables:

Limitations: Note that JSim import support for .m files would require a complete reimplementation of Matlab, so don't even think about it. Also note that only numeric data is transferred. Translating JSim MML source code into corresponding Matlab executable code would require a very significant additional programming effort, and is not anticipated at this time. It is possible, however, that Matlab binary .mat files may be supported for both import and export as some future time. If this is of pressing interest to you, please contact the JSim development team.

I4 Bullseye Data Format

This format is used by the I4 Imaging System for storage of static (2D) and dynamic (3D) cardiac bullseye data. JSim supports this format (both input and output) as part of its functional imaging facility which is currently under development. The recommended file suffix for this format is .bul .

Plug-in Data Formats

JSim also offers a data format plug-in feature that allows users to define support for other file formats within JSim.

Comments or Questions?

[This page was last modified 17Jul12, 9:22 am.]

Model development and archiving support at physiome.org provided by the following grants: NIH/NIBIB BE08407 Software Integration, JSim and SBW 6/1/09-5/31/13; NIH/NHLBI T15 HL88516-01 Modeling for Heart, Lung and Blood: From Cell to Organ, 4/1/07-3/31/11; NSF BES-0506477 Adaptive Multi-Scale Model Simulation, 8/15/05-7/31/08; NIH/NHLBI R01 HL073598 Core 3: 3D Imaging and Computer Modeling of the Respiratory Tract, 9/1/04-8/31/09; as well as prior support from NIH/NCRR P41 RR01243 Simulation Resource in Circulatory Mass Transport and Exchange, 12/1/1980-11/30/01 and NIH/NIBIB R01 EB001973 JSim: A Simulation Analysis Platform, 3/1/02-2/28/07.