4.2.1. Main Block

Replaces the main module in HSPsquared and provides for the outermost level of mHSP2 functionality.

4.2.1.1. locaMain.py

The main block for mHSP2 which is a replacement for HSPsquared main.

HSPsquared (HSP2) was completely modified and rearranged to create a main time loop with a sub-loop of operations/targets. The original HSPF logic is a main-loop of operations/targets and then a sub-loop for each target that is the time loop. Consequently, the time loop is run completely for each operation and target.

To couple HSPF to MODFLOW6, need to be able to break into the time loop at the beginning of each day. This Python module provides the replacement for HSP2main.py for coupling to MODFLOW 6. There are two main time loop versions within this module. The user controls which version is run by executing ..\coupledMain.py or ..\standaloneMain.py. Both “mains” use functions within this module. Function salocaMain is for standalone execution, and functions setUPPreTL and mainTL are for coupled (to MODFLOW 6) execution.

The message passing queue functions are only used when coupled mode execution is requested. Also the main block (if __name__ == main():) in this module is what is launcehd to start the independent process, spawned by the coupled controller and queue manager.

locaMain.AUTHKEY = b'authkey'

Authorization key for queue access.

This is not currently setup in a secure manner. If you plan on using pyHS2MF6 across a public or partially public network, then you should give some thought to the AUTHKEY and a means of securing this value. AUTHKEY needs to be encoded to a byte string.

locaMain.CLIENTHOST = '127.0.0.1'

Host machine for queue clients.

Clients are on the same ‘machine’ as the queue server when local descriptors like ‘localhost’ or ‘127.0.0.1’ are used.

locaMain.DAILY_DELT_STR = '1440'

DELT string representing daily, 24 hours * 60 minutes

locaMain.GFTAB_DICT = {}

Global FTABLE dictionary that stores the FTABLEs by FTABLE number

locaMain.GTS_DICT = {}

Global time series dictionary that stores the time series by SVOLNO key

locaMain.HOST = 'localhost'

Host machine for queue server.

Localhost means that the process are set-up to be on all the same machine. The queue server and the clients are in different processes on the same ‘machine’.

locaMain.KEY_ACT_IWAT = 'IWATFG'

Key value for simulating standard impervious water runoff

locaMain.KEY_ACT_PWAT = 'PWATFG'

Key value for simulating standard pervious water runoff

locaMain.KEY_ACT_RRHYD = 'HYDRFG'

Key value for simulating HYDR portion of reach reservoirs

locaMain.KEY_TS_PET = 'PETINP'

External time series key for input PET

locaMain.KEY_TS_PRECIP = 'PREC'

External time series key for precipitation

locaMain.MAP_TS_DICT = {}

Dictionary to map solution structure to time series.

Keys, tuple of (solution type, ID), and values [ SVOLNO, TMEMN, TVOLNO]

locaMain.PORT0 = 45492

Port number for the HSP2 queue.

Port numbers for queues need to be the same for each independent process for connection. Additionally, ports need to be opened in any firewall software even for local simulation.

locaMain.PORT1 = 45493

Port number for the MODFLOW 6 queue.

Port numbers for queues need to be the same for each independent process for connection. Additionally, ports need to be opened in any firewall software even for local simulation.

locaMain.PORT2 = 45494

Port number for global error handling and communications queue.

Port numbers for queues need to be the same for each independent process for connection. Additionally, ports need to be opened in any firewall software even for local simulation.

locaMain.QEND_MSG = ['End']

End of simulation message

locaMain.QINIT_MSG = ['Hello']

Queue intialization and checkin message

locaMain.QREADY_MSG = ['Ready']

Queue intialization and checkin message

locaMain.QUEUE_ERROR = ['Error']

Error message to put on queues for program termination

locaMain.QUEUE_TIMEOUT = 300.0

Queue wait timeout before error in seconds.

This is for the from pyMF6 queue and determines the mHSP2 wait time for receipt from MODFLOW 6.

class locaMain.QueueManager(address=None, authkey=None, serializer='pickle', ctx=None)

Create the queue manager class.

Needs to be at top level of the module so that is pickleable. Do not need anything here except to subclass SyncManager.

locaMain.QueueServerClient(ThisHost, Porter, CustomAuth)

Get a client connection a queue.

The connection is bidirectional and can use this connection for both get and put.

Parameters

• ThisHost (str) – host name. Should be ‘127.0.0.1’ for the same machine

• Porter (int) – the port number to listen on

• CustomAuth (str) – the custom authorization string

Returns

the client connection

Return type

locaMain.QueueManager

locaMain.SEQUENCE_DICT = {'IMPLND': [[100, 'ATMPFG'], [200, 'SNOWFG'], [300, ['IWATFG', 'IWATER']]], 'PERLND': [[100, 'AIRTFG'], [200, 'SNOWFG'], [300, ['PWATFG', 'PWATER']]], 'RCHRES': [[100, ['HYDRFG', 'HYDR']]]}

Holds the required calculation sequence by supported target type.

In HSPsquared, this is read in from the HDF5 file. Anyone can change edit the HDF5 file which opens the possibility of unintentional breakage of the program through incorrect editing of a portion of the HDF5 file. Consequently, this is hard coded here because to modify this sequence successfully, the code needs to be modified.

locaMain.SIMTIME_INDEXES = {}

Dictionary that holds simulation time indexes in case they happen to be different

locaMain.START_QUEUE_TO = 300.0

Queue wait timeout before error in seconds.

This is for program startup communications.

locaMain.SUPPORTED_ACTIVITIES = {'IMPLND': [['IWATFG'], ['IWATER']], 'PERLND': [['PWATFG'], ['PWATER']], 'RCHRES': [['HYDRFG'], ['HYDR']]}

Dictionary of supported activities

locaMain.TARG_DICT = {}

Dictionary with target types as keys and list of target ids as values

locaMain.TARG_IMPLND = 'IMPLND'

Key value for impervious land targets

locaMain.TARG_PERVLND = 'PERLND'

Key value for pervious land targets

locaMain.TARG_RCHRES = 'RCHRES'

Key value for reach/reservoir targets

locaMain.WriteQueueCheckToLog(qMF6)

Write queue checks to log file

Parameters

qMF6 (Queue) – the from MODFLOW6 queue

Returns

function status; 0 == success

Return type

int

locaMain.checkOpsSpec(allops, ucs, hdfType)

Checks the activities and operations desired and creates lists of target types.

Currently only PWATFG for PERLND, IWATFG for IMPLND, and HYDR for RCHRES are supported. Before going through the time and operations loops warn the user that only these will be implemented

Parameters

• allops (np.recarray) – operations listing from locaHSP2HDF5

• ucs (dict) – user control dictionary from locaHSP2HDF5

• hdfType (int) – type of HDF5 file; 0 == original format; 1 == new format

Returns

function status; 0 == success

Return type

int

locaMain.getDailySimTimeIndex()

Returns the daily simulation time index from the SIMTIME_INDEXES dictionary.

Must be called after setSimTimeIndexes

Returns

daily simulation time series index

Return type

pd.datetimeindex

locaMain.getTARG_DICT()

Get the global target dictionary.

Returns

TARG_DICT

Return type

dict

locaMain.initAllocTargStructures(sim_len)

Initialize all target structures including time series, flags, parameters, and initial values.

This takes care of initializing or allocating the simulation memory by creating np.recarrays of the simulation length.

Parameters

sim_len (int) – number of time steps in the simulation

Returns

function status; 0 == success

Return type

int

locaMain.mainTL(sim_len, tIndex, qHSP2, qMF6, hdfTyper)

The main time loop for mHSP2 in the coupled program.

Currently only 1440.0 minutes, or daily, is supported for sim_delt or a daily time step.

Parameters

• sim_len (int) – number of daily time steps in the simulation

• tIndex (pd.datetimeseries) – daily time index for simulation

• qHSP2 (Queue client) – from HSP2 queue

• qMF6 (Queue client) – from MODFLOW 6 queue

• hdfTyper (int) – type of HDF5 file; 0 == original format;0 1 == new format

Returns

function status, 0 == success

Return type

int

locaMain.metaChecks(start_dt, end_dt, num_pl, num_il, num_rr)

Check the remaining meta data items passed from the queue server as part of initial setup of a coupled simulation.

Requires that the input HDF5 file has already been read and processed into program structures.

Parameters

• start_dt (dt.datetime) – starting date time

• end_dt (dt.datetime) – ending date time

• num_pl (int) – number of PERLND

• num_il (int) – number of IMPLND

• num_rr (int) – number of RCHRES

Returns

function status; 0 == success

Return type

int

locaMain.nKEY_ACT_IWAT = 'IWATER'

New HDF5 format, key value for simulating standard impervious water runoff

locaMain.nKEY_ACT_PWAT = 'PWATER'

New HDF5 format, key value for simulating standard pervious water runoff

locaMain.nKEY_ACT_RRHYD = 'HYDR'

New HDF5 format, key value for simulating HYDR portion of reach reservoirs

locaMain.processArrayComm(iI, NpArray)

Process the array communication from MODFLOW 6.

Parameters

• iI (int) – current day, 0-based in the simulation

• NpArray (np.array) – the array of discharge to the surface from MODFLOW 6

Returns

function status; success == 0

Return type

int

locaMain.processInitMeta(PassList)

Process the initial metadata communication received from the queue server.

Call the various routines as required to read in all of the HDF5 information as required to completely set up the model and check start time, end time, number of pervious, number of impervious, and number of reach res

Parameters

• PassList (list) – list, L, of items - these are pickled and

• and following formats are required. (unpickled) –

  0. (str): model directory

  1. (str): FQDN pathname for HDF5 input file

  2. (dt.datetime): start time

  3. (dt.datetime): end time

  4. (int): number of PERLND

  5. (int): number of IMPLND

  6. (int): number of RCHRES

  7. (int): number of 2D cells, NCPL in MF6

  8. (str): FQDN pathname for RCHRES mapping dictionary

  9. (str): FQDN pathname for PERLND mapping dictionary

  10. (int): number of UZF cells in MF6

  11. (str): FQDN pathname for SPRING mapping dictionary

Returns

two items that provide the necessary information for continuing the simulation.

0. (int): return status; needs to be a member of hdfTyper - 0 or 1 for a success status

1. (str): hdfname - FQDN path for simulation HDF5 file. Empty string for error.

Return type

tuple

locaMain.processReadyComm(StringList)

Process the ready communication with external processes.

Parameters

StringList (list) – list of string items

Returns

function status; success == 0

Return type

int

locaMain.salocaMain(simdir, hdfname, saveall=False, reloadkeys=False)

Runs main HSP2 program in standalone mode.

Rewrite of original to make one main time loop

Parameters

• simdir (str) – verified model simulaton directory

• hdfname (str) – HDF5 filename used for both input and output.

• saveall (bool) – Saves all calculated data ignoring SAVE tables.

• reloadkeys (bool) – Regenerates keys, used after adding new modules.

Returns

function status, 0 == success

Return type

int

locaMain.setFlowLinks(linkdd, mldd, hdfType)

Setup the link structues, or routing, among targets.

This requires a combination of the defined mass links and schematic defined in the UCI file and ported to the HDF5 file.

At this point only linkages among “flow” types: PERLND; IMPLAND; and RCHRES are supported. Note that only RCHRES is supported as a receiving target - this means that a usable mass link and schematic link can only have RCHRES as the TVOL and that PERLND and IMPLAND can never be the TVOL.

Parameters

• linkdd (dict) – dictionary with schematic linkage among targets

• mldd (dict) – mass link definitions

• hdfType – type of HDF5 file; 0 == original format; 1 == new format

locaMain.setHRUAreas(linkdd)

Extract the HRU surface areas and store for post-processing.

This involves setting the areas for PERLND and IMPLND. These are only stored in the SCHEMATIC and LINKS sections of the inputs as the area factor or AFACTOR. So have to process through this linkdd which is ordered by reach to extract these values

Parameters

linkdd (dict) – dictionary of links with reaches as keys

Returns

function status; 0 == success

Return type

int

locaMain.setOutputSave(ucs, hdfType)

Extract the save specifications from the hdf5 storage structures and send to the target modules.

Parameters

• ucs (dict) – user control dictionary from locaHSP2HDF5

• hdfType (int) – type of HDF5 file; 0 == original format; 1 == new format

Returns

function status; 0 == success

Return type

int

locaMain.setParmsFlagsUCS(sim_delt, ucs, hdfType)

Transfer the parameter values and flags from the hdf file to our target modules.

Currently only “PWATFG”, “IWATFG”, and “HYDRFG” are supported. Note that these names have changed slightly under the new HDF5 file format and both naming conventions are supported.

Parameters

• sim_delt (float) – time step duration

• ucs (dict) – user control dictionary from locaHSP2HDF5

• hdfType (int) – type of HDF5 file; 0 == original format; 1 == new format

Returns

function status; 0 == success

Return type

int

locaMain.setSimTimeIndexes(allops, general, hdfType)

Sets the simulation time indexes from operational sequence in the hdf5 file.

At this time are only excepting daily time steps so this is all that will be returned. If a daily time step is not specified in the operational sequence, then an error will be thrown.

Parameters

• allops (np.recarray) – operations listing from locaHSP2HDF5

• general (dict) – GENERAL dictionary from locaHSP2HDF5

• hdfType (int) – type of HDF5 file; 0 == original format; 1 == new format

Returns

function status; 0 == success

Return type

int

locaMain.setTargDataTS(sim_len)

Set the input time series into the target structures.

Note that this only works with daily simulation time steps Time steps are stored in minutes so this should always be 1440.0 minutes or 1 day.

Parameters

sim_len (int) – number of time steps in the simulation

Returns

function status; 0 == success

Return type

int

locaMain.setUPPreTL(hdfname, saveall=False, reloadkeys=False)

Does all of the main mHSP2 setup for a coupled simulation.

Called prior to starting the main, time, loop.

Parameters

• hdfname (str) – HDF5 filename used for both input and output.

• saveall (bool) – Saves all calculated data ignoring SAVE tables.

• reloadkeys (bool) – Regenerates keys, used after adding new modules.

Returns

function status; also needs to be one of the hdfTyper

supported values, [0,1]; successful execution of this function means that either 0 or 1 is returned.

Return type

int

locaMain.writeOutputs(hdfname, tIndex, hdfType, IsCoupled)

Write out the outputs at the end of the simulation.

Parameters

• hdfname (str) – HDF5 file to output to

• tIndex (pd.DateTimeIndex) – time series index for outputs

• hdfType (int) – type of HDF5 file; 0 == original format; 1 == new format

• IsCoupled (bool) – is a coupled simulation?

Returns

function status; success == 0

Return type

int