[SciPy-User] TimeSeries and Milliseconds for aircraft data

[Ariel Rokem]

Hi everyone,

Gokhan is referring to this:

http://nipy.sourceforge.net/nitime/

For an exposition, see our Scipy conference proceedings paper (a pdf of
which can be found here:
http://argentum.ucbso.berkeley.edu/papers/Rokem2009Nitime.pdf).

We are still working on it. The intention of the library is to support
analysis of data from neuroscience experiments, because we are
neuroscientists, but so far, I don't think that we have made any design
decisions that would preclude other scientists from using our time-series
objects. In fact, the time-series objects we have designed support temporal
resolutions as fast as picoseconds (the representation of time is done in
int64, in order to avoid float-precision issues). It is still under
development and we have yet to make a release of this, but the code (in
development) is already available on github and the tests therein can direct
you on the possible usage:

http://github.com/fperez/nitime

Cheers,

Ariel

On Tue, Feb 23, 2010 at 7:23 AM, Gökhan Sever <gokhansever at gmail.com> wrote:

>
>
> On Tue, Feb 23, 2010 at 3:35 AM, Chris Jesse <
> chris.jesse at flightdataservices.com> wrote:
>
>> Hi All,
>>
>> I've got lots of timeseries data from aircraft sensors (such as altitude,
>> temperatures, landing gear up/down, airspeed..) which I was hoping to load
>> into a timeseries array. However, I've just read that it only supports time
>> resolutions down to a second. Is there a solution for my data which contains
>> samples from hundreds of different parameters at different frequencies from
>> 8Hz to 1/64Hz over an 8 hour period?
>>
>> I need to be able to interpolate between values, mask values when samples
>> are corrupt / outside of operational boundaries and store values as
>> accurately to the nearest 1/16th of a second if possible.
>>
>> Your help is much appreciated! Thanks,
>>
>> Chris
>>
>> _______________________________________________
>> SciPy-User mailing list
>> SciPy-User at scipy.org
>> http://mail.scipy.org/mailman/listinfo/scipy-user
>>
>>
> Hi Chris,
>
> Here at the University of North Dakota we work with aircraft measurements
> as well. Similar to your case depends on the campaign [1] sometimes the
> number of measured parameters reach over hundred. We use a QNX 4.25 based
> data acquisition system called M300 where all the probe data are connected
> in. M300 saves everything in a binary file which we later process using our
> ADPAA package [2] It is mostly IDL coded and mainly developed by Dr. David
> Delene [3] He is also the main responsible person for the aircraft
> measurements/deployments in our department.
>
> ADPAA produces files using a pre-determined ASCII data structure shown here
> [4] The system supports frequencies up to 100 Hz but typically the highest
> we sample is at 25 Hz. There might be exceptional cases where we might need
> higher sample rates. However 1-4 Hz is sufficiently enough when sampling
> clouds with cloud-aerosol microphysical instruments. (PCASP, FSSP, CCN) (I
> am disregarding instruments like holographic particle detectors for the
> simplicity of my response :) which we don't often get to fly with.)
>
> Anyway after some advertisement back to your question: in our system all
> the data are time-stamped with a main "Time [seconds]; UT seconds from
> midnight on day aircraft flight started" value. It simplifies data analysis
> a lot in my view where the Python comes into action actually. Once the ASCII
> files are created (we also output to NetCDF containers but I find easy to
> work on simple text files.) I read the data using NumPy and I use the
> MaskedArray module to deal with the missing values in the data. I publish my
> code along with data on the ccnworks [5] page where I mostly commit for my
> thesis using ground and airborne measurements and scientific Python
> ecosystem. For instance to read the ASCII data files (also known as NASA
> formatted) I use the generic script [6] As a part of my thesis I am working
> on this script [7] recently. I use the lab data [8] so far. In logn-fit.py I
> have a section where I do masking on the data like:
>
> mask02 = (pcasp.data['Time'] > 13800) & (pcasp.data['Time'] <= 14400)
>
> This is one of the most important part of the code --to extract the right
> portion of the required data. In this case I am interested in time periods
> based on the supersaturation values from the DMT-CCNC instrument. The next
> step will be to analyse the cloud base data. As far as I know there is no
> easy solution for this. What do PCASP probe shows when the aircraft is
> levelling below a cloud deck :) The only way I know is to mask manually and
> most logically using the common time-stamp in all data files.
>
> I remember Berkeley guys work on a project called NiPy but it is for
> neuro-scientists mostly and not much design consideration in it for airborne
> measurement scientists in mind :)
>
> I hope extra information doesn't bother you and let me know your opinions.
>
>
> [1] http://atmoswiki.aero.und.edu/atmos/citation/home
> [2] http://sourceforge.net/projects/adpaa/
> [3] http://aerosol.atmos.und.edu/
> [4] http://aerosol.atmos.und.edu/ADPAA/fileheader.html
> [5] http://code.google.com/p/ccnworks/
> [6] http://code.google.com/p/ccnworks/source/browse/trunk/nasafile.py
> [7]
> http://code.google.com/p/ccnworks/source/browse/trunk/thesis/part1/logn-fit.py
> [8]
> http://code.google.com/p/ccnworks/source/browse/trunk/thesis/part1/20090225_034214/09_02_25_03_42_14.lab.clean
>
> --
> Gökhan
>
> _______________________________________________
> SciPy-User mailing list
> SciPy-User at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-user
>
>


-- 
Ariel Rokem
Helen Wills Neuroscience Institute
University of California, Berkeley
http://argentum.ucbso.berkeley.edu/ariel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.scipy.org/pipermail/scipy-user/attachments/20100223/c44b59bf/attachment.html>