Hello Reader, here is this month’s iRODS news and developments!

If you’re facing an issue with iRODS you’re not sure how to solve, please do drop me a line; if I’ve come across a solution or seen something relevant elsewhere I’ll do my best to let you know. Or just drop me a mail to say ‘Hi’. Always nice to hear from people, particularly in these pandemic times!

I’d love your thoughts and feedback on how this newsletter could be better for you.

News

​TRiRODS: Towards Secure Supervisory Control and Data Acquisition (SCADA) with InfiniBand and iRODS​

​iRODS Globus Connector 4.2.8.1 is released.​

​Python iRODS Client v0.9.0 is released -This is well worth checking out - parallel transfer for 4.2.8 and later servers.

​iRODS Development Update: May 2021​

UGM 2021 Special Edition!

The 13th Annual #iRODSUGM is your chance to meet #iRODS users, Consortium members, and staff to discuss iRODS-enabled applications + discoveries, technologies powered by iRODS, and the future of iRODS and the iRODS Consortium Register Here​

Abstracts for talks + papers for the #iRODSUGM are due on May 15 (I know its past that, but if you’ve just found out, why not make a submission anyway?). They’re seeking presentations on the impact of iRODS on individuals, society, science, and systems & technology.

2021 will be a collection of pre-recorded + live talks w/ discussion. Additionally, June 11 will feature a troubleshooting session where participants can get 1:1 help with an existing or planned #iRODS installation or integration.

I’ll be there, do say hi on Slack!

Main Repository Activity

Lots of activity, it’s like a release is just around the corner…

Open Issues

​server SIGABRTs freeing invalid pointer when indexing plugin runs​

If you’ve not encountered the Indexing Plugin its a way to send all the metadata to an ElasticSearch database for speedier free text searching, than the default database imeta searches, especially when you get to milions of items of metadata.

​RBUDP transfer fails for largish files​

Do you use the UDP transfer method? Do you see the issue Tony reports? Chime in if so…

​remove -Wno-ignored-qualifiers​

Targeted at 4.2.10, if this affects you.

​iphymv error when moving replica into a composite tree​

Otherwise, I’d be inclined to think this has to do with the hierarchy resolution logic of the replication resource plugin getting tangled in the fact that this is a phymv, which is why we decided to enforce storage resources only for phymvs moving forward.

​difference in ‘schema_name’ in service account irods_environment.json​

This happens in the upgrade from 4.1.x to 4.2.x, not sure what implications it has, if any.

​non-service-account icommands are noisy when hosts_config.json not readable​

​ibun fails silently when extracting the contents of partially uploaded tar file​

​Add support for optional server configuration service endpoint​

In support for the creation of a cloud-native iRODS implementation, the server configuration needs to be moved to an external service for access on agent initialization.

Cloud native iRODS! Exiting!

​Investigate assertions made in test_iget_with_purgec​

​release activities for 4.2.10​

Confused? I was, this is actually just housekeeping and prep, not skipping 4.2.9 to go straight to 4.2.10!

​upgrade 4.1 to 4.2.8 on catalog consumer fails to migrate command scripts​

Custom command scripts in iRODS/server/bin/cmd/ should be moved to msiExecCmd_bin/ on upgrade, but dont seem to be - something to be aware of if you’re doing an upgrade. That being said, it might be specific to the reporters environment?

​Repeated istreams to new replicas of same data object results in various issues​

​*REG_PARAM in pep_database_mod_data_obj_meta_post sometimes string​

​msiExecCmd in acPostProcForModifyDataObjMeta causes data object modification to fail​

​need consistency in object type identifiers casing used by microservices​

​Allow comparison of columns with iquest​

​Voting on OPEN ignores pre-existing replicas not on default resource ​

Part of the long-running work into intermediate replicas.

​irods::plugin_context not serialized in pep_database_close_* PEPs​

​iRODS rule language parser(?) logging “Error updating cache.” ​

​maximum_number_of_concurrent_rule_engine_server_processes seems not to work​

​optimization/atomicity when manipulating resource hierarchies​

This issue discusses the situation where; >When performing ‘tree surgery’, there is a small window of time when a connecting client will get and use an ‘incorrect/incomplete’ resource hierarchy from the database.

​data okay, but rsync filling logs with ERROR when two providers in HA configuration (-4000)​

​imeta command returns 0 exit code on invalid input​

​iput using too much memory?​

​Define ACL meaning​

Yay documentation!

​inconsistent naming in ACL’s​

..That’s a paddlin’.

​ichksum -aK ignores status: down​

Often, this is what you’d want as an admin, but…

​replication resource stacktrace - INVALID_LEXICAL_CAST​

​release activities for 4.2.9​

Fingers crossed!

​Upgrade from 4.1.12 to 4.2.7 doesn’t update server_config.json when icat host FQDN on Ubuntu 18.04​

​prepareNonServerSharedMemory error​

TL;DR, Blame Systemd (possibly).

​Uploads during rebalance create extra replica​

​iadmin modresc rebalance failure​

​ files have >2 replicas, where one of them is zero length (but not marked dirty)​

Cor, three years ago! Looking like its fixed in 4.2.9’s intermediate replicas…

​irods connection on port 1247 to an azure server gets timed out, consider setting socket options in code.​

See 2533, below…

​Investigate TCP_KEEPALIVE on the server​

Worth checking this out if you do large transfers, or need to traverse firewalls which you might not control the timeout settings on.

Closed Issues

closed on - 2021-05-02 15:14:49 noisy logs - caught exception when replicating to a resource which doesn’t exist​

closed on - 2021-05-13 13:13:13 Undocumented new dynamic PEPs pep_api_replica_[open|close]_[pre|post|except|finally]​

closed on - 2021-05-13 21:17:06 parallel transfer in a network with firewalls drops ‘long idle’ connections​

See 2533, above…

closed on - 2021-04-21 19:47:19 iquest matches the select keyword when it shouldn’t​

closed on - 2021-05-13 12:11:05 iphymv (et al?) returns SYS_USER_NO_PERMISSION when it should be ENOENT (or equivalent)​

Some nice error code tidying here, very useful if you need to script iphymv.

closed on - 2021-04-22 11:48:54 resource_id information missing from izonereport json​

closed on - 2021-05-05 15:38:38 Add PUT KW during iput.​

closed on - 2020-07-06 22:57:31 remove CS_NEG_DONT_CARE as client/server SSL configuration option​

Turns out we do care (that this doesn’t work)!

closed on - 2021-05-06 14:41:56 Non-package install missing some dependencies​

Lots of good work on making the build asier recently.

closed on - 2021-05-06 13:29:22 ifsck gives boost error when filesystem contains a directory with no permissions for irods user​

Such as lost+found…

closed on - 2021-04-30 12:30:51 irodsAgent processes writing to an old log file​

closed on - 2020-09-15 02:24:44 deb package should be lintian clean​

closed on - 2020-09-15 02:25:10 get rpm build into state of rpmlint error-free​

Python iRODS Client Activity

Lots of activity here, for the release mentioned in the News section.

Open Issues

​Executing a rule requires reconnection / reauthentication to iRODS​

We’ve found that when executing multiple relatively fast (~10ms) rules in succession, or when performing any other PRC action after a PRC rule call, that the steps needed to re-open a connection to iRODS, negotiate and (PAM) authenticate, add significant performance overhead. In this case, a multiple of the duration of the rule execution itself.

​Backtick (`) encoding issue in iRODS XML protocol handling​

​Housekeeping (typos, whitespace, etc.)​

​Testing, versioning, packaging​

Closed Issues

Lots of stuff worked on and closed for v0.9.0!

closed on - 2021-05-14 20:10:28 Streamline testing​

closed on - 2021-05-14 20:26:52 iRODSSession objects not guaranteed to call cleanup( )​

closed on - 2021-05-14 20:26:39 Genquery fields for RULE_EXEC_* currently not provided in PRC​

closed on - 2021-05-05 14:46:10 PRC gives different serialized parameters than iCommand when a PEP is triggered​

closed on - 2021-04-29 14:54:01 How to iGet a collection using the python client?​

closed on - 2021-04-26 22:55:15 document connection timeout override​

closed on - 2021-05-14 20:15:16 Atomic operation on metadata errors with symbols in the value​

closed on - 2021-05-14 20:30:54 Metadata added by atomic metadata operations not visible during session​

closed on - 2021-05-14 20:33:38 Missing Exception class for code -130000​

closed on - 2021-05-14 20:27:46 ichksum equivalent?​

closed on - 2021-05-14 20:18:53 Provide parallel transfer for data object PUTs and GETs​

closed on - 2021-05-14 20:11:33 configure library for logging​

closed on - 2021-05-14 20:12:15 iRODSDataObject open( ) imposes incorrect opinions when accessing a replica by resource name​

closed on - 2021-05-14 20:21:12 Tests generally failing, and getting a Jenkins dot for the Python client​

NFSRods Activity

Nothing public…

icommands Activity

Open Issues

​showing metadata ID​

​