Time
S
Nick
Message
07:49 Virgile joined #dataverse
08:05 sc joined #dataverse
08:06 sc left #dataverse
08:51 Virgile joined #dataverse
10:55 pkiraly joined #dataverse
12:43 Virgile joined #dataverse
13:07 sivoais joined #dataverse
13:07 sivoais joined #dataverse
13:14 Virgile joined #dataverse
13:21 Virgile joined #dataverse
13:54 donsizemore joined #dataverse
13:56 donsizemore25 joined #dataverse
14:50 pdurbin joined #dataverse
15:45 pdurbin
There's talk of what to do with the server that iqlogbot is running on. It creates the logs at http://irclog.iq.harvard.edu/dataverse/today . It needs new hosting or needs to get shut down. Any thoughts on this?
15:47 Virgile
hello
15:47 Virgile
i find it useful..
15:48 pdurbin
me too :)
15:48 Virgile
it happened several times when I asked something, then had to log off, and on the next day i looked into the logs to have my answer
15:49 pdurbin
oh good, that's the whole idea :)
15:49 Virgile
:)
16:02 poikilotherm2
"Any thoughts on this?" Switch to Matrix. 🙈
16:03 pdurbin
Would we lose public logging? iqlogbot only knows IRC . :)
16:06 pkiraly
pdurbin, given that we are working accross the globe it happened sometime that I could reply to somebody when s/he was not available in the hope that s/he will check the log. Also I received messages via the log. So I vote to new hosting instead of shut it down.
16:07 nightowl313 joined #dataverse
16:10 poikilotherm2
Idea: create a logging matrix account, let it join IRC via Matrix bridge. Create Jenkins job to collect history every day, write to HTML and push to Github Pages
16:11 poikilotherm2
This could as well be used when we switch to Matrix, stepping away from IRC
16:13 poikilotherm2
Or use an Amazon Tiny (free) VM to log with the current software, fetch once a day via Jenkins and push to Github Pages (so the tiny instance is not overloaded))
16:14 pdurbin
pkiraly: I'm glad the logs are working for you too.
16:14 pdurbin
poikilotherm2: it's ok if we switch away from IRC . I just don't want to lose logging.
16:15 Virgile
@pdurbin +1
16:16 pdurbin
Inkscape has a nice chat service with public logs: https://chat.inkscape.org
16:23 Lincoln joined #dataverse
16:24 Lincoln
HI , I have a doubt with pyDataverse and custom metadata of dataset connection
16:24 pdurbin
Hi. What kind of doubt?
16:25 Lincoln
the custom metadata (in json format) works well with curl command but with pydataverse only citation metadata is visible
16:25 nightowl313
does anyone know if users get force logged out after inactivity in dataverse? And, is there a way to force log them out?
16:27 pdurbin
Lincoln: with pyDataverse, do you feed it the same JSON file that you would give to curl?
16:27 Lincoln
pdurbin: I use the same json file
16:28 pdurbin
nightowl313: that's a very good question. I may need to look into this soonish because I'm working on a "deactivate user" feature and I haven't thought much about what happens when users are logged in when they're deactivated. (But QA did, thankfully.)
16:29 pdurbin
Lincoln: huh, so does pyDataverse remove some fields? That would be odd. Let me see if Stephan is around.
16:30 nightowl313
we have a test user created for a periodic vulnerability scan, and the scan isn't running so we are trying to make sure that user is not active, and if we can force him to logout
16:30 pdurbin
Lincoln: I pinged him. What if you click the "Metadata" tab? Do you see all the fields you expect?
16:30 nightowl313
sorry to interrupt the conversation! =) i can always just restart the server
16:31 pdurbin
nightowl313: that will definitely kill the user's session. Yeah, let's see if we can help Lincoln and then pick this up in a bit.
16:33 Lincoln
pdurbin: if you click the metadata tab only citation metadata is visible but if "edit metadata" tab is clicked then you find also the other metadata s(including custom metadata) but is not filled
16:34 pdurbin
very strange
16:34 pdurbin
Can you please open an issue?
16:35 pdurbin
at https://github.com/gdcc/pyDataverse/issues
16:35 stefankasberger joined #dataverse
16:36 Lincoln
sure ill post it right away :) thank you all
16:36 pdurbin
sure, thanks for the heads up about this
16:37 pdurbin
nightowl313: I'm asking about the session thing in #dv-tech
16:37 nightowl313
thank you!
16:37 stefankasberger
@Lincoln: that's right. pyDataverse does not support custom metadata so far, only the default version of citation, geospatial, socialscience and journal.
16:39 Lincoln
stefankasberger: thank you .. Shall i still raise an issue?
16:39 Lincoln
*on github
16:40 stefankasberger
yes, please.
16:41 stefankasberger
then i will link it in the re-factor models issue #102 (https://github.com/gdcc/pyDataverse/issues/102)
16:41 Lincoln
sure
16:41 stefankasberger
Thanks!
16:41 stefankasberger
How important is this feature to you?
16:42 stefankasberger
I am asking, cause maybe you want to implement it and create a PR to close the issue yourself.
16:42 stefankasberger
i will do this in the summer, so expected release somewhere around august/september.
16:42 stefankasberger
will be part of 0.5.0
16:45 Lincoln
As dataverse database will be linked with other application , it would be rather nice to have this feature rather than overwriting(customizing) in the default metadata
16:45 Lincoln
(*for my application)
16:45 nightowl313
got another question ... we are trying to change the main navigation of DV to accomodate our branding requirements, which means moving all of the main mav items to the ASU main nav menu ... in dataverse, once a user is logged in, the "Login" menu changes to a user menu with other options and the Login menu goes away .. is that handled via primefaces, or javascript?
16:47 Lincoln
@stefankasberger: If it helps i have used following script. ds=Dataset() ds_filename="test.json" ds.from_json(read_file(ds_filename))
16:48 Lincoln
if you do ds.get() then the custom metadata is gone
16:49 pdurbin
nightowl313: primefaces (sorry for the cross-talk)
16:52 nightowl313
no, sorry, I'm interrupting you all! =D
16:53 stefankasberger
yes, it only imports known data structure, which is the default one from Dataverse.
16:54 Lincoln
stefankasberger: thanks again .. @nightowl313 ..please continue the chat
16:56 pdurbin
nightowl313: so, there is no way to force a user to logout. The session length is configurable though. Let me go find the docs.
16:57 pdurbin
:LoginSessionTimeout
16:57 pdurbin
https://guides.dataverse.org/en/5.3/installation/config.html#loginsessiontimeout
16:58 pdurbin
poikilotherm2: ooh, another example of web.xml there. Thanks for opening that Payara issue.
16:58 nightowl313
oh, a session timeout
16:58 nightowl313
LoginSessionTimeout
16:58 pdurbin
yep
16:59 nightowl313
perfect .. sorry i didn't see that =)
16:59 pdurbin
no worries, many docs :)
17:00 poikilotherm2
pdurbin: not looking for 5 minutes, huge traffic in the room.
17:00 poikilotherm2
pdurbin: you're welcome (for opening the issue)
17:00 nightowl313
as for the navigation branding, I was looking for a way to change the javascript to change the new menu item, but I think what I will need to do is change the dataverse elements to use the custom styles for our branding
17:01 pdurbin
nightowl313: I'd suggest opening a GitHub issue.
17:01 nightowl313
okay, will do ... thanks!
17:32 poikilotherm2
pdurbin: Q?
17:33 poikilotherm2
I paste it here, maybe you can send an answer later (leaving for construction site now)
17:35 poikilotherm2
In https://github.com/poikilotherm/dataverse/blob/8b2b47845106121e321cbb2be9a4be2454fdd989/src/main/java/edu/harvard/iq/dataverse/search/IndexServiceBean.java#L856-L858 and following, Solr gets fields from metadata blocks in two ways: 1. for a static defined field you need to put inside schema.xml plus 2. into a dynamic field, if the field is facetable. Why is that? Isn't there already the static field? Why push to
17:35 poikilotherm2
multivalue string field, too?
17:42 poikilotherm2
This is especially strange when reading about faceting in the Solr guide. It makes sense to use two fields, one for search, one for faceting to avoid issues with text fields:
17:42 poikilotherm2
"If you want Solr to perform both analysis (for searching) and faceting on the full literal strings, use the copyField directive in your Schema to create two versions of the field: one Text and one String. The Text field should have indexed="true" docValues=“false" if used for searching but not faceting and the String field should have indexed="false" docValues="true" if used for faceting but not searching. (For m
17:42 poikilotherm2
information about the copyField directive, see Documents, Fields, and Schema Design.)"
17:42 poikilotherm2
https://solr.apache.org/guide/8_8/faceting.html
17:42 poikilotherm2
But we are not doing that as far as I can see from the schema.xml
17:43 poikilotherm2
The dynamic fields are still docValues = true + indexed = true
17:44 poikilotherm2
(Just like the normal field, you need to add to the schema.xml when activating a metadata block)
17:51 poikilotherm2
And I'm puzzled that we use a string field for facets, when integers etc allow range facets in their datatypes.
17:51 donsizemore joined #dataverse
17:51 poikilotherm2
It's all a bit strange...
18:04 Lincoln
hi again, I am currently stuck with connection of ceph cluster system (s3)with dataverse. I am not sure of my custom-endpoint-url and would like your help to fix this
18:07 poikilotherm2
Domain style or subpath style
18:11 poikilotherm2
Oh. Stupid question. RADOS S3 CEPH Gateway is path style only
18:14 poikilotherm2
Can you access your S3 endpoint with other tools like s3api?
18:14 poikilotherm2
CERN has a big Ceph-based S3 service, so maybe their tutorials might help in ensuring you actually can access the endpoint. http://cephdocs.s3-website.cern.ch/user/aws-cli.html
18:14 Lincoln
i can access my s3 account with s3cmd (sorry I am just a beginner with ceph)
18:19 poikilotherm2
Great. Depending on your Ceph installation, they either let you access buckets with path style or by using virtual domains
18:19 poikilotherm2
Often virtual domains are not used as those are a burden to maintain
18:19 poikilotherm2
So it's very likely you will need to configure Dataverse to use path style access
18:20 poikilotherm2
See https://guides.dataverse.org/en/latest/installation/config.html#s3-storage-options
18:31 Lincoln
poikilotherm2: its kind of stupid to ask in my .s3cfg file the website_endpoint looks like http://%(bucket)s.s3-website-%(location)s.amazonaws.com i am confused how to deal with this
18:32 poikilotherm2
Uhm what? That's not a Ceph S3 endpoint, that Amazon S3.
18:32 poikilotherm2
What's the endpoint you're using with s3cmd?
18:33 poikilotherm2
(No need to post it cleartext here, maybe obfuscate)
18:40 Lincoln
its something like cephmon1.*-institution_name-*:*-portnumber-*
18:48 Lincoln
@ poikilotherm2: I solved it. Thank you.
19:01 pdurbin
poikilotherm2: thanks for helping Lincoln. Do you still have Solr questions? You seem to have different Solr questions in #dv-external. :)
19:04 poikilotherm2
Na, Same questions but I had the idea to load balance, hopefully not taking too much of your precious Dataverse time
19:06 pdurbin
It's all good. Busy day here in this channel today. :)
19:16 donsizemore joined #dataverse
19:17 donsizemore
it may interest @pkiraly to learn that I picked back up my podman all-in-one test suite implementation today. today's run resulted in "Tests run: 137, Failures: 9, Errors: 7, Skipped: 8"
19:17 pdurbin
nice
19:18 donsizemore
it's in podman on Ubuntu 20.04 over SSD, but I blame me for selecting the "experimental ZFS support" box when I installed the OS
19:18 donsizemore
ZFS performance on SSD got better when I switched podman to fuse-overlay, but it's still fairly sluggish
19:21 donsizemore
@pdurbin I'm pestering you in Slack where I can paste a question with formatted text
19:28 pdurbin
thanks, I see it
20:35 donsizemore
@pdurbin [WARNING] Tests run: 146, Failures: 0, Errors: 0, Skipped: 8
20:35 pdurbin
wat
20:35 pdurbin
that's great!
20:46 donsizemore
next to troubleshoot repo and branch passing, then I may be ready to let somebody else pick it apart
20:48 pdurbin
awesome
21:53 pdurbin left #dataverse
22:21 dataverse-user joined #dataverse
