Time
S
Nick
Message
07:10 poikilotherm joined #dataverse
07:22 jri joined #dataverse
08:40 stefankasberger joined #dataverse
09:56 stefankasberger joined #dataverse
10:26 jri joined #dataverse
10:34 jri joined #dataverse
11:10 pdurbin joined #dataverse
11:10 pdurbin
mornin' all
11:47 stefankasberger
morning. :)
11:47 stefankasberger
@pdurbin: a question regarding, of course, the api.
11:48 stefankasberger
how must language attributes be passed? i have no example dataset, where the attribute naming, the structure and the values can be seen.
12:25 stefankasberger
?
12:25 pdurbin
stefankasberger: sorry, I'm not quite sure what you mean. The UI of Dataverse can be in different languages but the metadata for datasets is always in English.
12:38 donsizemore joined #dataverse
13:05 stefankasberger
there is an attribute "language" in the metadata. (see https://demo.dataverse.org/dataset.xhtml?persistentId=doi:10.70122/FK2/P2VLO5) . how can this be created/updates programmatically via API ?
13:38 jri
Hello :) I got an issue when trying to plug our Shibboleth: after the login the shib.xhtml redirect shows nothing dans I'm not logged in. Dataverse log says "The SAML assertion for "Shib-Identity-Provider" was null. Please contact support." . Alright but when I browsing https://my-dataverse/Shibboleth.sso/Session I can see all informations needed by dataverse (including "Identity Provider: https://federation-test.sciences-po.fr/idp/sh
13:38 jri
ibboleth"). Did I miss something ?
13:38 jri
How does Dataverse is doing to get the value of this attribute ? (HTTP Headers, env vars, etc)
13:41 donsizemore
@jri any chance you could send me your apache config?
13:41 donsizemore
@jri ssl.conf or whatever you named it
13:41 pdurbin
stefankasberger: hi, sorry I was dropping my daughter and her friend off at sailing camp. Still there?
13:41 donsizemore
@jri and are you certain your identity mgmt group is sharing the R&S attribute set with your dataverse host?
13:42 donsizemore
@pdurbin for many a stormy wind shall blow, ere Jack comes home again!
13:43 pdurbin
:)
13:44 pdurbin
donsizemore: I merged the ec2 scripts. Thanks again for that.
13:45 pdurbin
jri: environment variables are used.
13:48 jri
donsizemore: you can see it here: https://0bin.net/paste/OXtGW6ZNnZDw4UVQ#iT6lsJ-feec8HNHpgvAO2yXrGJRaC9RSnNYnRfYPWQF (I just removed my domain). It's very simple, but you have to know that we also have Nginx reverse proxys before Apache that handle https (HTTPS Request -> Nginx Reverse Proxy --HTTP --> Apache ----> Shibboleth SP/dataverse)
13:48 donsizemore
@pdurbin i'm picking up where i left off with munin. i think after this next run i'll be ready for you to test the branch and let me know what needs tweaking
13:49 donsizemore
@jri your virtualhost listens on 80 but your servername specifies 443? (i get the nginx part, but is mod_shib happy with that?)
13:49 pdurbin
donsizemore: awesome. And do you have any advice for modifying YAML with a script? Do you have a favorite tool? Somethink like jq maybe?
13:50 donsizemore
@pdurbin i only modify yaml with vim, but i can do some webstalking
13:50 pdurbin
My use case is to try to keep up with main.yml as it evolves but to have a few values that I want to always insert or update. If that makes sense.
13:51 pdurbin
jri: does it work if you remove Nginx? Temporarily, I mean.
13:52 jri
Hmm I can't really do that, I don't have a public IP address and access to firewalls to directly publish my dataverse server
13:52 pdurbin
bummer
13:52 jri
And I need to publish it because we use renater federation
13:52 pdurbin
poikilotherm: help!
13:53 poikilotherm
Someone rang a bell?
13:53 pdurbin
poikilotherm: jri needs your new auth stuff :)
13:53 poikilotherm
Hehehe
13:53 pdurbin
jri: have you been following what poikilotherm is up to?
13:54 jri
No :) What is it ?
13:55 poikilotherm
https://github.com/IQSS/dataverse/issues/5974
13:56 donsizemore
@jri in case this helps https://github.com/nginx-shib/nginx-http-shibboleth/issues/27
13:58 jri
Oh I think I found my mistake!
13:58 jri
missing attributePrefix="AJP_" in ApplicationDefaults in shibboleth2.xml
13:58 jri
my bad :(
13:58 donsizemore
yes you need that
13:59 jri
Sorry...
13:59 pdurbin
It's so long since I set up shib I would probably forget that too. :)
14:00 jri
I literaly passed my day on this... I had to talk with you guys to find my way :)
14:01 pdurbin
:)
14:03 pdurbin
donsizemore: where should the sushi file go instead of /tmp?
14:05 donsizemore
assuming multiple unprivileged users, anywhere beneath ~/ should be fine
14:05 donsizemore
@pdurbin want to test my 91_munin branch and lemme know what i need to fix? basic install is in place
14:07 pdurbin
donsizemore: sure! My other thought was to ask the operating system for a tmp directory from Java. Something unique each time.
14:08 pdurbin
donsizemore: one of the things I want, now that the ec2 scripts are in dataverse-ansible, is the ability to spin up an arbitrary dataverse-ansible branch. Should I create an issue for this?
14:09 donsizemore
shore
14:10 donsizemore
and i can understand your wanting said switch ;)
14:10 pdurbin
I'm so lazy. :)
14:11 pdurbin
poikilotherm: oh, I'm talking to Slava about auth for weblate. This might interest you too, donsizemore. Do you have any opinions on how people should be able to auth?
14:13 donsizemore
seamlessly
14:13 poikilotherm
Github maybe?
14:13 donsizemore
GitHub and Goog are great starts
14:13 pdurbin
Slava just linked me to https://docs.weblate.org/en/weblate-2.4/admin/auth.html
14:14 stefankasberger
yes.
14:14 pdurbin
poikilotherm: I was thinking... you **could** run an auth service for the Dataverse community. Maybe it could be used for http://weblate.dataverse.org and https://jenkins.dataverse.org and services we haven't even dreamed up yet. :)
14:15 poikilotherm
He. The same thing came to my mind, too :-D
14:15 poikilotherm
Would be nice for testing
14:15 poikilotherm
And demos
14:15 pdurbin
stefankasberger: I don't see anything about language at https://github.com/IQSS/dataverse/blob/v4.15.1/scripts/api/data/dataset-create-new-all-default-fields.json but the easiest way is to go to https://demo.dataverse.org and create a dataset with whatever metadata fields, publish it, and they click "metadata export" and download it as JSON .
14:16 pdurbin
poikilotherm: I already mentioned the idea to Kevin but as I was talking about it I started thinking "Do I really want yet another account?" and I'm not sure that I do. But it would be good for testing, etc like you said.
14:17 poikilotherm
We could offer Github and Google auth in it
14:17 poikilotherm
No need for a new account
14:22 pdurbin
poikilotherm: for both Weblate and Jenkins?
14:22 pdurbin
poikilotherm: oh, you mean into the auth service, probably
14:22 poikilotherm
Yeah
14:23 pdurbin
One thing I'm interested in is having profile pages for members of the Dataverse community. I don't know if having an auth service gets us closer to that or not.
14:23 pdurbin
donsizemore: I created that issue and added it to your column.
14:23 pdurbin
poikilotherm: do you want a column on my board? :)
14:24 poikilotherm
Go ahead :-D
14:25 pdurbin
poikilotherm: done: https://github.com/orgs/IQSS/projects/3#column-5999026
14:27 pdurbin
poikilotherm: any more thoughts on 6000? Danny opened it back up. I was thinking about removing it from my board. It's something I want but I want other stuff more.
14:28 poikilotherm
Go ahead :-)
14:28 poikilotherm
I'm still working on it
14:28 poikilotherm
Still trying to figure out where to place the autocomplete function
14:28 pdurbin
Thanks, removed. I'm willing to wait until next Chrismas for it. Or the one after.
14:29 pdurbin
Right, you had all sorts of JSF questions.
14:29 pdurbin
And I'm not a JSF expert but I can try to answer them if you still have those questions. Or different questions.
14:29 poikilotherm
He
14:30 poikilotherm
I still don't fully understand why things are picked from different places...
14:30 pdurbin
poikilotherm: did I give you an account on our AWS account? I forget.
14:30 poikilotherm
Yeah, you did
14:30 pdurbin
Perfect. I hereby authorize you (and donsizemore) to spin up stuff for Slava. :)
14:31 poikilotherm
?
14:31 poikilotherm
What kind of stuff
14:31 pdurbin
I don't know. :)
14:31 poikilotherm
LOL
14:31 poikilotherm
As long as Danny pays the bill... :-D
14:32 pdurbin
exactly
14:32 pdurbin
I'll give Danny a heads up.
14:41 poikilotherm
pdurbin do you think it would make more sense from an architecture view to place the Java function for the autocomplete values in DatasetField instead of DatasetFieldType?
14:43 pdurbin
poikilotherm: I don't have a strong preference honestly. It's the kind of thing we can move around during code review if someone objects to where you put it.
14:44 poikilotherm
Ok.
14:46 stefankasberger
i worked it out (started the same path independently from your answer). so the language import now works too in pyDataverse (in the next release). :)
14:46 pdurbin
stefankasberger: great! Please feel free to create a pull request to updata that "all files" JSON file, which obviously doesn't have all the fields in it! :)
14:46 poikilotherm
I'm outta here guys. Read you all tomorrow
14:47 pdurbin
update*
14:59 Slava1 joined #dataverse
15:01 Slava1
Hi guys! We need some Amazon account to test our Kubenetes pipeline.
15:02 pdurbin
Slava1: hi! What do you want your username to be?
15:03 andrewSC joined #dataverse
15:03 Slava1
dans
15:04 pdurbin
Slava1: can you send me the email address on Skype?
15:09 pdurbin
Slava1: thanks, I sent you two emails. You'll have to change the password when you first log in.
15:12 Slava1
Great, I'm in!
15:12 pdurbin
perfect
15:37 donsizemore joined #dataverse
16:32 donsizemore
@pdurbin i lied about testing my 91_munin branch... that's the one a sleepily committed straight to master. but if you'd like to test the 93 branch with the ec2 -a flag? (A is for ansible)
16:35 pdurbin
donsizemore: cool but what does DA stand for in DA_BRANCH?
16:36 pdurbin
dataverse-ansible?
16:40 donsizemore
yis
16:40 donsizemore
i'm happy to rename any variables
16:41 donsizemore
as a college intern i inherited perl using all sorts of horrible variable names: creamy_nougat and a number of unmentionables
16:41 pdurbin
heh
16:41 pdurbin
no, it's fine
16:42 pdurbin
I've been meaning to look into which base_url sample data uses.
16:43 pdurbin
"siteUrl_stripped" it looks like
16:43 pdurbin
but this doesn't work well for me when I have ssl enabled
16:45 pdurbin
and my dev2 certs
16:45 pdurbin
ssl.CertificateError: hostname 'ec2-54-144-69-232.compute-1.amazonaws.com' doesn't match 'dev2.dataverse.org'
16:45 pdurbin
that's from `python3 create_sample_data.py`
16:46 donsizemore
i think you can specify that? if not i can make it so
16:46 pdurbin
so maybe we should just use http://localhost:8080 instead?
16:47 pdurbin
Here's "set siteUrl in dvconfig.py": https://github.com/IQSS/dataverse-ansible/blob/97831ed8b87b431ba328e133ced31c363a27994e/tasks/sampledata.yml#L42
16:47 pdurbin
Anyway, for now I'm running create_sample_data manually, afterwards.
16:51 pdurbin
Ok, I'm trying the new 93_ec2_branch_switch. Going for a walk and will cross my fingers.
16:55 donsizemore
i can hard-code it to localhost:8080 and leave it until somebody complains ;)
17:12 pdurbin
donsizemore: I was also thinking about changing it on the dataverse-sample-data side. Not sure what I want.
18:17 jri joined #dataverse
18:25 pdurbin
donsizemore: do you want to hear about munin stuff?
18:37 donsizemore
shore
18:37 donsizemore
all i did so far was plop the package and plugins in place
19:03 pdurbin
donsizemore: should `rpm -qa | grep -i munin` show something?
19:05 pdurbin
Because munin doesn't seem to be installed? Even though this ran: "TASK [dataverse : install munin and dependencies] ***"
19:05 donsizemore
was it skipped? (i left the default set to false in "public" group_vars)
19:06 donsizemore
change this to true: munin: install: false
19:07 pdurbin
donsizemore: bah! thanks!
19:08 donsizemore
(sorry, i forgot to mention that. turn a number of things on in the "test/" group_vars that i leave off externally)
19:08 donsizemore
^^ i turn
19:08 pdurbin
donsizemore: is this a case where I can run this task after the fact? With a tag or whatever?
19:16 donsizemore
it has a tag "munin"
19:16 donsizemore
so in theory yes
19:17 pdurbin
And the README tells me how? :) I'll go look.
19:19 donsizemore
heh. it doesn't, but i can add that
19:19 donsizemore
just give it --tags=munin
19:19 pdurbin
Do I need all this stuff too? --connection=local -v -i dataverse/inventory dataverse/dataverse.pb -e dataverse/defaults/main.yml
19:20 donsizemore
yes. also make that --tags "munin"
19:21 pdurbin
ok :)
19:25 pdurbin
donsizemore: --tags worked! Thanks!
19:27 donsizemore
how about munin?
19:31 pdurbin
oh that
19:31 pdurbin
let me looks
19:31 pdurbin
it installed somethin
19:32 pdurbin
donsizemore: looking good! https://ec2-52-91-77-202.compute-1.amazonaws.com/munin/
19:33 pdurbin
donsizemore: Should I be worried about this? [WARNING]: Unable to find '/tmp/munin-monitoring/plugins/jvm' in expected paths (use -vvvvv to see paths)
19:33 donsizemore
huh? worked fine in vagrant
19:34 donsizemore
are the plugins in place? /usr/share/munin/plugins/j*
19:34 pdurbin
oh wait wait, wrong window :)
19:35 pdurbin
I do have /usr/share/munin/plugins/jstat__gccount
19:35 pdurbin
and friends
19:36 pdurbin
hmm, but not on that munin page I linked?
19:36 donsizemore
i noticed
19:36 pdurbin
Do they have to be enabled?
19:36 donsizemore
apparently
19:37 donsizemore
pffft "To enable a plugin, what needed to be done is essentially making a symlink (symbolic link) from the physical plugin file located in /usr/share/munin/plugins to the active plugins directory located in /etc/munin/plugins "
19:38 pdurbin
Yeah it looks like that directory is full of symlinks
19:38 donsizemore
i'll fix that
19:38 pdurbin
cool, and they I can re-run --tags "munin" right?
19:38 pdurbin
then*
19:38 donsizemore
yes
19:39 pdurbin
\o/
20:00 donsizemore
try it now?
20:04 pdurbin
/etc/munin/plugins/jstat__gccount: symbolic link to `/usr/share/munin/plugins/jstat__gccount'
20:04 pdurbin
thanks!
21:39 donsizemore joined #dataverse
23:56 jri joined #dataverse
