The Resource logo

The Resource

Archives
Subscribe
September 20, 2022

The September 2022 Edition of The Resource

Hello Reader, here is this month’s iRODS news and developments!

If you’re facing an issue with iRODS you’re not sure how to solve, please do drop me a line; if I’ve come across a solution or seen something relevant elsewhere, I’ll do my best to let you know. Or just drop me a mail to say ‘Hi’. Always nice to hear from people, particularly in these pandemic times!

I’d love your thoughts and feedback on how this newsletter could be better for you.

News

TRiRODS Aug 2022 - Interns

Covered last month but worth mentioning again - the YouTube of the TriRods session showcasing the RENCI Interns work.

More Unofficial Documentation!

This month we added tips on using clientUserName, Singularity and tweaked the repo to run tests and auto-deploy on merge.

More contributions welcome!

Main Repository Activity

Open Issues

​Consider adding feature test macros​

​CMake: CMAKE_POSITION_INDEPENDENT_CODE should be overridable​

​CMake: Do not set CMAKE_EXPORT_COMPILE_COMMANDS​

​ichmod should not be allowed to bypass the permission model​

Potentially controversial, this one! The proposal is to remove the ability for the original uploaded to always be able to restore their own permissions via ichmod. It’s always been an outlier in terms of normal ACL management approaches, but ichmod -M as a rodsadmin will continue to work that’s better from an audit perspective, I would say.

​Support interactive PAM authentication plugin​

Let's reopen this to track work on integrating the interactive PAM plugin as it seems we do not currently have one of those.

​iget verify checksum​

Not sure what the requestor was asking here.

​Tokens related to remote rule execution are possibly unused​

A superficial examination of the iRODS core code reveals two quantities which are set, but never used:
"remExec" is inserted as either a prefix or wrapper (depending on context) for the rule text in the remoteExec and smsi_remoteExec functions . But across the codebase, it seems never to get scanned-for, let alone used, despite the uneasy indication that it is somehow central to how rule text gets parsed (and indeed appears to be the only input to rsExecMyRule indicating the rule text is meant for remote execution).
"execCondition" is a key set in the execMyRuleInp_t structure by addKeyVal -- also in functions remoteExec and smsi_remoteExec -- but also is apparently never scanned-for or used.
iRODS Version, OS and Version
iRODS 4.2.11 and 4.3.0
(... and all the way back into iRODS 3.x and iRODS-Legacy)

​irods-icommands package missing dependencies on Almalinux 8​

The client library was not installed alongside the iCommands package. Seems there's just a little gap in the packaging for this platform that needs to be filled in. This problem is not exhibited on Ubuntu 20.04.

​Segmentation fault in ils and tickets access​

Still being investigated, but the Issue author was kind enough to allow me to add their Singularity / Modules implementation into the Unofficial iRODS docs.

​Is these Iget and Iput speed correct?​

The issue author ran the icommands on the same server as the iRODS system, so I suspect Linux was caching the files in memory.

​Document detached mode for unixfilesystem resources​

Looking forward to this one.

​Investigate adding .editorconfig​

EditorConfig is meant to be an mostly-universal way of telling editors and IDEs how to treat certain files. This can be used to provide defaults for indent width, line endings, and some code formatting directives.

​EditorConfig - this was news to me and I added it to my personal repos.

​Investigate .git-blame-ignore-revs​

​Unused .cpp files in irods rule language rule engine source tree​

I wonder what happened to those functions/features?

​Enhancement Request: automate regression reporting from issues​

Currently, issues are reported on whichever version the reporter finds them on, and are tested on that version and (usually) the latest version. When a fix is found, usually a test is added to ensure regression doesn't occur in the future.
It would be nice to have these tests run against prior versions (say back to 4.2...something), so that the issue can be updated with which versions are susceptible to the bug.

The reply points to We should add a framework (or at least a set of instructions) to facilitate using git bisect to find the commit which introduced a bug. in the Development Framework (which I’ve not managed to get back to and really should).

​demoResc SYS_RESC_DOES_NOT_EXIST​

TL;DR, if you delete demoResc;

The solution is to msiSetDefaultResc to a resource that exists in the system, and the message should go away.

​Can’t link using irods_resource_plugin-unixfilesystem​

​iRODS fails to transfer large file between zones if receiving zone has SSL enabled and sending zone has no SSL​

This appears to be parallel transfer, when SSL is enabled on zone A and disabled on zone B.

Reported on 4.2.10.

Closed Issues

Closed on - 2022-09-09 18:29:07 Clang-Format: Don’t align trailing comments​

Closed on - 2022-09-09 18:28:55 Disable clang-tidy check: readability-avoid-const-params-in-decls​

Closed on - 2022-09-07 18:58:36 clang-tidy complains about array to pointer decay whenever irods_error macros are used​

Closed on - 2022-08-30 10:59:40 Clang Format GitHub action fails due to unknown key​

Closed on - 2022-09-01 21:00:35 psqlodbc_irodsServer_irods log spam in /tmp​

Closed on - 2022-08-26 07:00:34 client_user option in icommand ​

I have to admit, this was a neat trick I didn’t know, so I added it to the Unofficial iRODS Documentation.

Administrator can get permissions acting like client_user to put data into his/her collection and the data owner is the client_user.
If there is way to do the same thing in icommand?

And there was;

The admin can also alias as any user via the 'clientUserName' environment
variable.
in the example we can just export clientUserName=tom to solve the problem.

This is kind of like sudo but only for ACL’s and only if you are already a rodsadmin!

Closed on - 2022-08-26 15:28:30 add binary package installation instructions to README​

Users that simply would like to installand use the icommands on their workstation would benefit from binary package installation instructions. These instructions could be added to the README file.

If you’re pointing your users at this, remember that there is no guarantee of compatibility between versions, so remind them to install the version you’re running on the servers. If you are the user, check with imiscsrvinfo once you’ve installed to check the remote system is running the same version.

If you didn’t want to take a chance on the wrong version, you can ask the server. For example, from the command line (replace localhost with the server name);

``` root@irodsServer:/# echo “hello” | nc localhost 1247 | egrep -a rods

rods4.2.10

```

Closed on - 2022-08-29 20:51:24 reFuncDefs.hpp is a private header, inaccessible when compiling plugins such as the Python RE plugin​

Closed on - 2022-08-24 02:26:17 Remove pyparsing python module​

Closed on - 2022-08-16 13:54:55 Should that inheritance only affects files added to the directory after the inheritance bit is set?​

No, this is the intended behavior.
Inheritance affects new data objects and subcollections...

Closed on - 2022-08-24 00:51:04 Remote configuration reloading​

Targetted for 4.3.1

The configuration reload feature doesn't cover any use cases with multiple instances.
Given that, it is desirable to add this to irods-grid so that it may be invoked across the zone, something like irods-grid reload all

Closed on - 2022-09-01 20:45:22 Davrods and Dynamic Policy Enforcement Points​

this is scoped for 4.3.1 as a Documentation exercise

Closed on - 2022-08-23 23:10:56 Remove .gitmodules​

Closed on - 2022-09-09 18:28:29 Merge filesystem.tpp with filesystem.hpp​

Closed on - 2022-09-09 18:36:53 Refactor logging API for static analysis tools and documentation​

Closed on - 2022-09-10 02:15:13 Add GitHub workflow for clang-tidy​

Closed on - 2022-08-21 19:03:33 iRODS server​

Some documentation about how to get started adding code to iRODS.

Closed on - 2022-09-10 02:30:16 Add lazy evaluation of format arguments to 4.3 logging API​

Closed on - 2022-08-31 23:41:25 remote( ) microservice should honor INST_NAME​

Targetted for 4.2.12, but reported on 4.3.0

Execute some iRODS native rule text remotely when a plugin other than the native rule plugin is first in the rule_engines[] part of the server configuration.
This should be possible by using INST_NAME tag in the hints argument to remote( ), and should cause only the targeted RE instance to see the request.

Closed on - 2022-08-26 04:12:30 metadata.hpp contains definitions that should be in a cpp file somewhere.​

Closed on - 2022-08-21 19:11:36 irods setup fails on centOS 7​

The reporter was trying to run iRODS built from an old copy of master, which was close to 4.3.0 pre-release.

Closed on - 2022-08-22 17:30:52 Update get_script_output_single_line() to use Boost.Process​

Closed on - 2022-08-22 17:31:45 Add settings refresher to main server​

Some settings require a server restart to apply.
If the facility exists as a thread in the main server process, it could update the settings in the main server process. If the facility exists as a forked process, it might be a bit harder to implement this.
new advanced setting to determine how often to run... default... 600 seconds?

Closed on - 2021-12-03 20:49:07 SQL generated by rebalance is too long for GenQuery (too many resources)​

Closed on - 2022-08-26 01:26:48 www.irods.org has different hostname in certificate​

The website is currently an auto-redirect to irods.org (without www), where the cert is valid. Like you say, the other browsers forward without complaint, and then certify correctly. Safari catches the intermediate hop and complains.
EFF's Certbot will begin issuing wildcard certs in January 2018 - and we will be in line when they are rolled out. This issue should resolve itself at that time.

Python iRODS Client Activity

Open Issues

​omitting admin flag as obj.metadata() arg causes error​

In PRC v1.1.4 (perhaps? likely? before), if you don’t call obj.metadata with the admin flag you get an error as it’s expected by the function.

​Expose create and modify times for metadata​

Allthough metadata items can be queried and filtered for create_time/modify_time columns, the higher level class iRODSMeta does not expose these fields. Consequently, they are not available when fetching metadata from collections and data objects (and resources and users).
Can these time fields be added?

​On Windows, path normalization replaces “/” with “"​

​irods_ssl_verify_server set to “cert” apparently not applied​

Whether "irods_ssl_verify_server": "cert" is in the irods_environment.json file or as an argument to iRODSSession (irods_ssl_verify_server="cert" or ssl_verify_server="cert"), it does not appear to be applied. In a high availability arrangement where the connection is passed to an iRODS server with a different hostname, setting the verification to "cert" is required and works fine with the iCommands. When a connection is made after a PRC session is instantiated, however, the error below is seen.

Closed Issues

Closed on - 2022-09-01 15:57:46 add status property to resource class​

Addressed sufficiently by https://github.com/irods/python-irodsclient/pull/381

icommands Activity

Closed Issues

Closed on - 2022-08-21 19:13:43 usr directory error​

Closing - no action needed here.

externals Activity

Closed Issues

Closed on - 2022-09-03 14:52:38 20.04 Ubuntu LTS Build Failure (clang: wrong gcc default)​

Clang fails to build under a fresh instance of Ubuntu 20.04 LTS
Suggested fix
Add update-alternatives as part of install_prerequisites.py
Or
Warn that this needs to be done at the end of install_prerequisites.py when gcc-9 is default

It is documented in one place;

This is part of the readme... Do those instructions need updating?
https://github.com/irods/externals#ubuntu-18-ubuntu-20-and-debian-11

but not in the other;

Ah, that documentation has it! --- I was using this documentation:
https://docs.irods.org/4.3.0/getting_started/installation/#non-package-install-run-in-place

Closed on - 2022-08-21 19:14:27 Build error in Ubuntu 18.04​

Sorry for the trouble - yes, most people are not building their own externals and so we've not spent as much time keeping that documentation as polished for multiple releases.
I recommend looking at 4.2.11 documentation - it's mostly the same as 4.2.3 (linked), but some things have moved.
make sure you're building the 4-2-stable branch (git checkout 4-2-stable)
no need for make -j - each build target is already parallelized, and the dependencies themselves are defined in the Makefile
you can try make server first.. this subset of targets are the requirements for the irods-server. additional plugins may require some of the other make targets.
As a separate approach, consider a regular packaged install first, to see what success would look like. Building everything from source is definitely a tough first leap.

YODA Activity

Open Issues

​[FEATURE] Data Access Password expiration notification​

When a Data Access Password has expired a WebDAV client will throw an authentication error, this might lead to unneeded support calls when researchers forget they have to set a new password themselves.
Describe the solution you'd like
Send an automatic notification to the user X hours (configurable for the instance) before a Data Access Password expires. The existing notification functionality in 1.8 seems suitable for this.
Describe alternatives you've considered
The only alternative I can think of is a dedicated Yoda WebDAV client that can show a more appropriate error message.

If you think someone else would appreciate this newsletter, they can sign up at https://theresource.metadata.school/​

One yak was shaved in the making of this newsletter.

Don't miss what's next. Subscribe to The Resource:
Powered by Buttondown, the easiest way to start and grow your newsletter.