Waveform parametrization module.
scwfparam is a SeisComP3 module that computes
in real-time or offline. It includes a process scheduler and handles reprocessing of data in a smart way. It supports ShakeMap XML output as documented in the ShakeMap manual each time a new set of data is available.
When the module is not started in offline mode, the processing of events is scheduled following the configured rules. Parameters that influence the scheduling are:
The wake-up interval specifies when the scheduler is called to check if a process is about to be started or stopped. The default is 10 seconds.
The scheduler checks then all scheduled jobs, adds a job to the processing queue if the next run time is not in the future and removes all scheduled jobs with timestamps in the past. The process queue contains all jobs that are about to be executed. Because waveform acquisition is a time- and memory-costly operation only one process can run at a time. Once a process finished, the next process in the queue is executed (if any). When a process is started, it fetches the latest event parameters (origin time, magnitude, location).
To add processes to the scheduler, the module distinguishes two cases:
When a new event or an event update is received which does not have an associated process yet, a new process is created. The event time (Origin[Event.preferredOriginID].time) is used to build the default schedule according to wfparam.cron.delayTimes.
for each time in wfparam.cron.delayTimes:
add_cron_job(process, Origin[Event.preferredOriginID].time + time)
If a process for an event already exists, the next run time is the current time plus wfparam.cron.updateDelay. Before adding this job to the scheduler the application checks if the next scheduled runtime is at least wfparam.cron.updateDelay seconds after the new run time. If not, a new job is not addded to the scheduler. Pseudo code to illustrate the strategy is given below.
event_updated(event):
p = process_for_event(event)
# The schedule for process p could be {T1,T2,T3,T4}
now = get_current_time()
next_run = now + wfparam.cron.updateDelay
# Process currently suspended?
if isEmpty(p.schedule):
p.schedule.add(next_run)
elif (p.schedule[0] - next_run) > wfparam.cron.updateDelay:
p.schedule.prepend(next_run)
else:
# Do nothing, ignore the event update
pass
The processing can be divided into the following steps:
The channel is considered to be processed if the last step succeeded.
If wfparam.output.waveforms.enable is set to true all processed waveforms are stored in the configured output directory wfparam.output.waveforms.path. The naming convention of a channel MiniSEED file is:
[EventDateTime]_[net]_[sta]_[loc][cha]_[filter][order]_[freqs].mseed
If wfparam.output.waveforms.withEventDirectory is set to true, an event directory with the eventID is created additionally where the channel files are stored under.
Either:
/path/to/waveforms/file1.mseed
/path/to/waveforms/file2.mseed
...
or
/path/to/waveforms/eventid/file1.mseed
/path/to/waveforms/eventid/file2.mseed
...
The MiniSEED file contains uncompressed float 4096 byte records.
Example:
| Event time | 2011-11-21 08:30:00 Network: CH |
| Station | SNIB |
| Location | _ _ |
| Channel | HGZ |
| Filter | hi-pass |
| Order | 2 |
| Corner frequencies | 0.025 |
| Filename | 20111121083000_CH_SNIB_HGZ_HP2_0.025.mseed |
scwfparam can make use of the database schema extension for strong motion parameters.
In order to prepare the database the extension schema must be applied. The database schema is installed in share/db/wfparam/*.sql. Login into the database backend and source the .sql file corresponding to the used database backend.
In order to enable scmaster to handle messages containing objects for strong motion parameters load the dmsm (data model strong motion) plugin as follows in scmaster.cfg:
plugins = ${plugins}, dmsm
scmaster must be restarted to activate the plugin.
To activate scwfparam to send messages with strong motion objects, set
wfparam.output.messaging = true
in scwfparam.cfg.
The ShakeMap XML is generated according the documentation of version 3.5 if wfparam.output.shakeMap.enable is set to true.
Below an example is given of an event XML and a station XML. The data was generated from a playback and does not describe a real event.
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<!DOCTYPE earthquake SYSTEM "earthquake.dtd">
<earthquake id="gfz2011oasp" lat="38.916" lon="40.0711"
depth="10.3249" mag="5.80361" year="2011"
month="7" day="19" hour="14" minute="54"
second="21" timezone="GMT"
locstring="tst2011oasp / 38.916 / 40.0711"
/>
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<!DOCTYPE earthquake SYSTEM "stationlist.dtd">
<stationlist created="" xmlns="ch.ethz.sed.shakemap.usgs.xml">
<station code="JMB" name="JMB" lat="42.467" lon="26.583">
<comp name="BHZ">
<acc value="0.0175823522" flag="0"/>
<vel value="0.0265134476" flag="0"/>
<psa03 value="0.0177551343" flag="0"/>
<psa10 value="0.0179450342" flag="0"/>
<psa30 value="0.0507100318" flag="0"/>
</comp>
</station>
<station code="BUD" name="BUD" insttype="STS-2/N"
lat="47.4836" lon="19.0239">
<comp name="BHZ">
<acc value="0.0018418704" flag="0"/>
<vel value="0.0012123935" flag="0"/>
<psa03 value="0.0019287320" flag="0"/>
<psa10 value="0.0033152716" flag="0"/>
<psa30 value="0.0027636448" flag="0"/>
</comp>
</station>
<station code="ANTO" name="ANTO" lat="39.868" lon="32.7934">
<comp name="BHZ">
<acc value="0.0322238962" flag="0"/>
<vel value="0.0250842840" flag="0"/>
<psa03 value="0.0326696355" flag="0"/>
<psa10 value="0.0621788884" flag="0"/>
<psa30 value="0.0903777107" flag="0"/>
</comp>
</station>
<station code="GNI" name="GNI" lat="40.148" lon="44.741">
<comp name="BHZ">
<acc value="0.0760558909" flag="0"/>
<vel value="0.0273735691" flag="0"/>
<psa03 value="0.0818660133" flag="0"/>
<psa10 value="0.1230812588" flag="0"/>
<psa30 value="0.1682284546" flag="0"/>
</comp>
</station>
</stationlist>
Running scwfparam offline with a multiplexed miniseed volume, an event xml and an inventory xml file. A hi-pass filter of 0.1hz (10secs) is used. Processing starts immediately and the application finishes when processing is done. The scheduler is disabled in offline mode.
scwfparam --offline -I vallorcine.mseed \
--inventory-db vallorcine_inv.xml \
--ep vallorcine.xml -E "Vallorcine.2005.09.08" \
--lo-filter 0.1 --hi-filter 0
Running for a given event with scheduling enabled. Only the given event will be processed.
scwfparam -I arclink://localhost:18001 -E gfz2011oeej \
-d mysql://sysop:sysop@localhost/seiscomp3
For running in real-time it is enough to add the module to the client list of the trunk package in seiscomp config.
Running with remote Arclink server
To use a remote Arclink server it is enough to configure the record stream with -I:
scwfparam --offline -I vallorcine.mseed \
--inventory-db vallorcine_inv.xml \
--ep vallorcine.xml -E "Vallorcine.2005.09.08" \
-I "arclink://arclink.ethz.ch:18002"
Note that the default acquisition timeout of 30 seconds might not be enough to get all the requested data. If necessary, increase the value with parameter wfparam.acquisition.initialTimeout. This can also be reached on command line:
scwfparam --offline -I vallorcine.mseed \
--inventory-db vallorcine_inv.xml \
--ep vallorcine.xml -E "Vallorcine.2005.09.08" \
-I "arclink://arclink.ethz.ch:18002" \
--wfparam.acquisition.initialTimeout=300
Running with remote Seedlink server
To use a remote Seedlink server it is enough to configure the record stream with -I:
scwfparam --offline -I vallorcine.mseed \
--inventory-db vallorcine_inv.xml \
--ep vallorcine.xml -E "Vallorcine.2005.09.08" \
-I "slink://geofon.gfz-potsdam.de:18000"
scwfparam inherits global options.
Type: int
Default value of total time window length in seconds if wfparam.magnitudeTimeWindowTable is not specified. This times window includes wfparam.preEventWindowLength. Default is 360.
Type: list:string
Magnitude dependent time window table. The format is "mag1:secs1, mag2:secs2, mag3:secs3". If a magnitude falls between two configured magnitudes the time window of the lower magnitude is used then. No interpolation is performed. Magnitude outside the configured range are clipped to the lowest/highest value. Example: "3:100, 4:200, 5:300"
Type: int
The pre event time window length in seconds. Default is 60.
Type: int
The maximum epicentral distance in km of a station being considered for processing. This value is used if wfparam.magnitudeDistanceTable is not specified. Default is 400.
Type: list:string
Analogue to wfparam.magnitudeTimeWindowTable but instead giving a time window, the distance in km is specified. Example: "3:400, 4:450, 5:500"
Type: double
Relative saturation threshold in percent. If the absolute raw amplitude exceeds X% of 2**23 counts the station will be excluded from processing. This assumes a 24bit datalogger. Default is 80.
Type: double
Specifies the STA length in seconds of the applied STA/LTA check. Default is 1.
Type: double
Specifies the LTA length in seconds of the applied STA/LTA check. Default is 60.
Type: double
Specifies the minimum STALTA ratio to be reached to further process a station. Default is 3.
Type: list:double
Specifies a list of damping values (in percent) for computation of the relative displacement elastic response spectrum. Example: "5,10,15" Default is 5.
Type: int
Specifies the number of natural periods for computation of the relative displacement elastic response spectrum between Tmin and Tmax. If fixed is given then a fixed list of periods is used. Default is 100.
Type: double
Specifies the minimum period (Tmin) in seconds for computation of the relative displacement lastic response spectrum. Default is 0.
Type: double
Specifies the maximum period (Tmax) in seconds for computation of the relative displacement elastic response spectrum. Default is 5.
Type: boolean
Enables/disables aftershock removal (Figini, 2006; Paolucci et al., 2008) Default is true.
Type: boolean
Enables/disables pre-event cut-off. A hardcoded sta/lta algorithm (with sta=0.1s, lta=2s, sta/lta threshold=1.2) is run on the time window defined by (expected_P_arrival_time - 15 s). The pre-event window is hence defined as [t(sta/lta =1.2) - 15.5s, t(sta/lta =1.2) - 0.5s]. Default is true.
Type: list:string
Default is 0:0.2;0.8fNyquist,3:0.1;0.8fNyquist,5:0.05;0.8fNyquist,7:0.025;0.8fNyquist.
Type: boolean
Enables/disables deconvolution. If a channel does not provide full response information it is not used for processing. Default is true.
Type: double
Defines the magnitude tolerance to completely reprocess an event with respect to the last state. Default is 0.5.
Note
wfparam.streams.* Defines the white- and blacklist of data streams to be used. The rules to decide if a stream is used or not are the following: ** 1. if whitelist is not empty and the stream is not on the whitelist, don't use it, ok otherwise ** 2. if blacklist is not empty and the stream is on the blacklist, don't use it, ok otherwise ** Both checks are made and combined with AND. Either whitelist or blacklist contains a list of patterns (wildcard allowed as * and ?), eg GE..*.*, , GE.MORC..BH? Each stream id (NET.STA.LOC.CHA) will* be checked against the defined patterns.
Type: list:string
The stream whitelist
Type: list:string
The stream blacklist
Type: boolean
Defines if a linear spacing or logarithmic spacing between Tmin and Tmax is used. The default is a linear spacing. The logarithmic spacing will fail if either Tmin or Tmax is 0. Default is false.
Note
wfparam.filter.* Parameters of the 1st stage filter.
Type: int
Specifies the order of the 1st stage filter. Default is 4.
Type: double
Specifies the frequency of the 1st stage hi-pass filter. If this parameter is equal to 0 the hi-pass filter is not used. If suffix "fNyquist" is used then the value is multiplied by the Nyquist frequency of the data to get the final corner frequency of the filter. Default is 0.025.
Type: double
Specifies the frequency of the 1st stage lo-pass filter. If this parameter is equal to 0 the lo-pass filter is not used. If suffix "fNyquist" is used then the value is multiplied by the Nyquist frequency of the data to get the final corner frequency of the filter. Default is 40.
Note
wfparam.sc.* Parameters of the 2nd stage (sensitivity correction) filter.
Type: int
Specifies the order of the 2nd stage filter. Default is 4.
Type: double
Specifies the frequency of the 2nd stage hi-pass filter. If this parameter is equal to 0 the hi-pass filter is not used. If suffix "fNyquist" is used then the value is multiplied by the Nyquist frequency of the data to get the final corner frequency of the filter. Default is 0.025.
Type: double
Specifies the frequency of the 2nd stage lo-pass filter. If this parameter is equal to 0 the lo-pass filter is not used. If suffix "fNyquist" is used then the value is multiplied by the Nyquist frequency of the data to get the final corner frequency of the filter. Default is 40.
Type: boolean
Enables non-causal filtering in the frequency domain. Default is false.
Type: double
Defines the cosine taper length in seconds if non-causal filters are activated applied on either side of the waveform. If a negative length is given 10 percent of the pre-event window length is used on either side of the waveform. Default is -1.
Type: double
The length of the zero padding window in seconds applied on either side of the waveform if non-causal filters are activated. If negative, it is computed following Boore (2005) as 1.5*order/corner_freq and applied half at the beginning and half at the end of the waveform. Default is -1.
Type: int
Specifies the interval in seconds to check/start scheduled operations. Default is 10.
Type: int
Specifies the maximum allowed idle time of a process before removed. The idle time is calculated if no further processing is scheduled and computes as: [now]-lastRun. Default is 3600.
Type: boolean
Enables/disables updating of a cron log file. This file will be created at ~/.seiscomp3/log/[appname].sched and contains information about the scheduled events and the processing queue. The file is updated each n seconds, where n = wfparam.cron.wakeupInterval. Default is true.
Type: int
Specifies the delay in seconds to delay processing if a new authoritative origin arrives for an event. Default is 60.
Type: list:int
Specifies a list of delay times in seconds relative to event time to trigger the processing. When the first origin of an event arrives this list is used to construct the crontab for this event. Example: "60, 120, 300, 3600"
Type: int
Specifies the initial acquisition timeout. If the acquisition source (e.g. Arclink) does not respond within this threshold with waveforms, the request is discarded. Default is 30.
Type: int
Specifies the acquisition timeout when waveforms are being transfered. If no new waveforms arrive within this threshold, the request is aborted. This is important if a Seedlink connection is configured which can block the application for a very long time if at least one requested channel has no data. Seedlink does not finished the request until all data has been sent. When data will arrive for a particular channel is not known. Default is 2.
Type: bool
Enables messaging output which creates objects of the StrongMotionParameters data model extension (defined by SED) and sends them to scmaster. In order to save the objects to the database, scmaster needs to load the dmsm plugin and the corresponding database schema must be applied. The default message group is AMPLITUDE. To change this group redefine connection.primaryGroup. Default is false.
Type: boolean
Enables/disables the output of processed waveforms. Default is false.
Type: string
Specifies the waveform output path. This parameter is only used if wfparam.output.waveforms.enable is true. Default is @LOGDIR@/shakemaps/waveforms.
Type: boolean
Enables/disables the creation of an event directory (named with eventID) when storing the processed waveforms. This parameter is only used if wfparam.output.waveforms.enable is true. Default is false.
Type: boolean
Enables/disables the output of spectra (psa, drs). The output format is a simple ASCII file where the first column is the period and the second column the corresponding value. Default is false.
Type: string
Specifies the spectra output path. This parameter is only used if wfparam.output.spectra.enable is true. Default is @LOGDIR@/shakemaps/spectra.
Type: boolean
Enables/disables the creation of an event directory (named with eventID) when storing the spectra. This parameter is only used if wfparam.output.spectra.enable is true. Default is false.
Type: boolean
Enables/disables ShakeMap XML output. Default is true.
Type: string
Specifies the ShakeMap XML output path. This is only used if wfparam.output.shakeMap.enable is set to true. Default is @LOGDIR@/shakemaps.
Type: string
Specifies the path to a script that is called whenever a new ShakeMap XML is available. The script is called with 3 parameters: EventID, modified ShakeMap eventID, path to event directory (where input/event.xml and input/event_dat.xml lives). The event files are not deleted by the application. The ownership goes to the called script.
Type: boolean
Enables/disables synchronous or asynchronous script calls. If enabled, be careful to not spend too much time in the script. The application is blocked while the script is running. Default is true.
Type: boolean
If enabled the maximum PGV, PGA, PSA03, PSA10 and PSA30 of both horizontal components is used in the final output. Otherwise each component is saved. Default is false.
show help message.
show version information
Use alternative configuration file. When this option is used the loading of all stages is disabled. Only the given configuration file is parsed and used. To use another name for the configuration create a symbolic link of the application or copy it, eg scautopick -> scautopick2.
Load given plugins.
Run as daemon. This means the application will fork itself and doesn't need to be started with &.
Enable/disable self-shutdown because a master module shutdown. This only works when messaging is enabled and the master module sends a shutdown message (enabled with --start-stop-msg for the master module).
Sets the name of the master-module used for auto-shutdown. This is the application name of the module actually started. If symlinks are used then it is the name of the symlinked application.
Sets the name of the master-username of the messaging used for auto-shutdown. If "shutdown-master-module" is given as well this parameter is ignored.
Time span in hours after which objects expire
EventID to calculate amplitudes for
EventParameters (XML) to load
Verbosity level [0..4]. 0:quiet, 1:error, 2:warning, 3:info, 4:debug
Increase verbosity level (may be repeated, eg. -vv)
Quiet mode: no logging output
Limits the logging to a certain component. This option can be given more than once.
Use syslog logging back end. The output usually goes to /var/lib/messages.
Path to lock file.
Send log output to stdout.
Debug mode: --verbosity=4 --console
Use alternative log file.
Overrides configuration parameter connection.username.
Overrides configuration parameter connection.server.
Overrides configuration parameter connection.timeout.
Overrides configuration parameter connection.primaryGroup.
A group to subscribe to. This option can be given more than once.
Overrides configuration parameter connection.encoding.
Sets sending of a start- and a stop message.
Test mode, no messages are sent
List all supported database drivers.
The database connection string, format: service://user:pwd@host/database. "service" is the name of the database driver which can be queried with "--db-driver-list".
The configmodule to use.
Load the inventory database from a given XML file.
Do not use the database at all
Filter order
High-pass filter frequency
Low-pass filter frequency
Sensitivity correction filter order
Sensitivity correction high-pass filter frequency
Sensitivity correction low-pass filter frequency
Do not connect to the messaging and to the database
Dump the configuration and exit
Dumps all received records (binary) to [eventd].recs