Time
S
Nick
Message
07:37 juancorr joined #dataverse
12:16 stefankasberger joined #dataverse
12:43 donsizemore joined #dataverse
12:43 donsizemore
@pdurbin i saw what i was doing wrong just as i had to leave yesterday. will fix eye promise
12:44 pdurbin
great! I haven't looked. Sorry. Me no speak Ansible.
12:45 pdurbin
Can I pick your brain about https://dataverse.org/metrics ?
12:46 pdurbin
welcome, stefankasberger
13:00 stefankasberger
hi!
13:01 stefankasberger
my name is Stefan Kasberger, and I am the DevOp of AUSSDA - The Austrian Social Science Data Archive. I am developing right now a Python package for the usage of Dataverse (pyDataverse, https://github.com/AUSSDA/pyDataverse) , and am looking for some contact regarding the existing python dataverse client. I was looking for contact information regarding Robert Liebowitz, but did not find any. Does anyone know, who is the best person t
13:01 stefankasberger
my name is Stefan Kasberger, and I am the DevOp of AUSSDA - The Austrian Social Science Data Archive. I am developing right now a Python package for the usage of Dataverse (pyDataverse, https://github.com/AUSSDA/pyDataverse ), and am looking for some contact regarding the existing python dataverse client. I was looking for contact information regarding Robert Liebowitz, but did not find any. Does anyone know, who is the best person
13:02 donsizemore
@pdurbin sure thing, what about metrics?
13:02 pdurbin
Hi! Sorry, you're getting a little cut off. Robert did the initial work but he's not involved anymore.
13:03 donsizemore
@stefankasberger I recognize a few of the committers but I bet IQSS would be delighted with some pull requests =)
13:04 pdurbin
Hmm, we still give credit to Robert at http://guides.dataverse.org/en/4.11/api/client-libraries.html#python which is good but perhaps we should make it more clear that he isn't actively involved in dataverse-client-r anymore.
13:04 pdurbin
donsizemore: yes, pull requests, even just to fix up that "client libraries" page, are very welcome!
13:05 donsizemore
@pdurbin i've already put a bug in Andrey's ear. he's on CoRE2 lockdown until the community meeting, though
13:05 stefankasberger
I already started a parallel development, cause our purpose is more based on writing transformation scripts to prepare data for the API import.
13:05 pdurbin
Are you importing with JSON or XML ?
13:06 stefankasberger
For me it would be great to just have someone with whom I can talk about the package architecture, API stuff and the approach in general. and how active the development is, and if the scope is maybe compatible.
13:06 stefankasberger
I use JSON .
13:08 pdurbin
Whoops, I said "r" when I meant "python".
13:08 stefankasberger
it was easier for me at the beginning to just write some API requests, and i thing also the usage of XML made it tricky for our purpose.
13:08 stefankasberger
it was easier for me at the beginning to just write some API requests, and i think also the usage of XML made it tricky for our purpose.
13:09 pdurbin
Sure. I hope you aren't too annoyed by our crazy JSON format. :)
13:09 stefankasberger
when i remember correctly, but the first inspection led to the decision to just start it from scratch (and anyway, its my first python package, i work only since 3 months with dataverse, so it was a good opportunity to get a little bit deeper in both).
13:10 stefankasberger
whats the crazy part about it? :)
13:10 pdurbin
Well, it's pretty verbose. All the "primitive" and "compound" nonesense that you shouldn't have to think about.
13:13 pdurbin
stefankasberger: here is the background on dataverse-client-r and how it isn't very actively maintained anymore: https://groups.google.com/d/msg/dataverse-community/lD24w0d3D88/GAh0OPKuAAAJ
13:14 stefankasberger
hahaha. yes, thats true. seems as i have adapted quite fast to it. :)
13:15 pdurbin
A pull request was made by https://github.com/pattisdr recently, so that's good.
13:17 pdurbin
I'm not sure the best way to get in touch with the OSF developers.
13:19 stefankasberger
@pdurbin: thanks, that was helpful. good to see, that others have the same problem (file uploading without curl).
13:21 pdurbin
Yeah. Various problems. Here's where OSF pulls in their fork of dataverse-client-python: https://github.com/CenterForOpenScience/osf.io/blob/23efbff9f6d40a9b43c5315c4b7b9e0628e86817/addons/dataverse/requirements.txt#L4
13:22 pdurbin
stefankasberger: maybe you or I could leave a comment at https://github.com/CenterForOpenScience/dataverse-client-python/pull/4 . What do you think?
13:23 pdurbin
Hmm, or maybe this one would be better since it hasn't been merged yet: https://github.com/CenterForOpenScience/osf.io/pull/8939
13:24 stefankasberger
i use the native api for download and upload of dataverses, datasets and files. @pdurbin: would you recommend doing it otherwise?
13:28 pdurbin
I think you're doing it right. SWORD is another possibility but it's more limited. The "S" in SWORD is for simple. You can't do as much.
13:29 pdurbin
So do you want me to leave a comment on that pull request? I'm happy to.
13:35 stefankasberger
so we/I can get in touch with her? or for another purpose?
13:35 pdurbin
Well, she is the one who hacked on the code most recently. A couple weeks ago. She might have some thoughts on it.
13:36 stefankasberger
sure. thats great. you can link to me on github (skasberger).
13:37 pdurbin
Sure, I'll @mention you. Another thought is that you could start fresh if you want. Parallel development, like you said. I could create a new repo under IQSS called dataverse-client-python-new or whatever so you can design from scratch.
13:39 pdurbin
The thing about dataverse-client-python is that is was created so long ago (DVN 3 days) that the only API we had was SWORD. In Dataverse 4 we have tons of "native" APIs that aren't being used in that Python library. No one has had the time or energy to add them.
13:39 stefankasberger
i have already created an own repo: https://github.com/AUSSDA/pyDataverse
13:40 stefankasberger
but thanks for the offer. maybe in the future this will be the way to got, but right now i am just happy to have my first package online and to exchange with others about my experiences.
13:41 stefankasberger
step by step. :)
13:41 pdurbin
One way to get others interested in your library would be for us to list it along side the older one at a future version of http://guides.dataverse.org/en/4.11/api/client-libraries.html . Do you want us to?
13:44 stefankasberger
I will have some deadline stress until end of next week, then i have planned to announce the first release.
13:45 pdurbin
Ok, no rush. Here's the 4.11 version of that page: https://github.com/IQSS/dataverse/blob/v4.11/doc/sphinx-guides/source/api/client-libraries.rst
13:45 stefankasberger
but in general: yes. but am afraid if i upload something not mature, that a lot of issues, PR or Emails return, which i will not be able to deal with in the next three weeks (will be at DANS together with Slava). at the end of april it gets better, were i planned to go outside a bit with the package.
13:46 pdurbin
Ah, you know Slava. Please tell him to stop by this channel now and then. :)
13:47 stefankasberger
we have met so far two times, but it always was interesting and fun. will do. :)
13:49 pdurbin
Cool. In this channel we've been talking about an idea for a Dataverse hackathon in Europe and I'd love to hear what he thinks. And what you think. And what other people in Europe think, like, uh, juancorr and pmauduit and maybe others (please help fill in the "who's who" in the topic of this channel).
13:52 pdurbin
stefankasberger: can you please show Slava this tweet? https://twitter.com/poi_ki_lo_therm/status/1093148857079382019
14:04 donsizemore joined #dataverse
14:07 stefankasberger
I am not really into java software dev, and am really now to the dataverse univers, but in general i love hackathon and have organized several in the past.
14:07 stefankasberger
have sent slava the tweet.
14:08 pdurbin
Thanks! I've never organized a hackathon. I participated in one at the Dataverse Community Meeting a couple years ago but it turned more into a "can anyone get a Dataverse dev environment working" fest. :/
14:09 pdurbin
stefankasberger: can you please send Slava a link to this "2020 Dataverse Hackathon in Europe README" as well? https://docs.google.com/document/d/1pbr0zsqMsl6NIAJCULUTTchMOsch2CAJX14_qQ2zFPE/edit?usp=sharing
14:30 donsizemore
@pdurbin @pameyer did we ever make any headway in that glassfish-not-wanting-to-die install snafu? i'm bumping into that for whatever reason
14:31 stefankasberger joined #dataverse
14:32 pameyer
@donsizemore I narrowed things down a tiny bit in the glassfish in docker not staying started; but still don't have a great understanding
14:33 stefankasberger
done too. :)
14:33 pdurbin
We discussed it during sprint planning yesterday and it got pulled into the sprint.
14:33 pdurbin
stefankasberger: thanks!
14:33 pameyer
I'm pretty sure I changed `asadmin restart-domain` to `asadmin stop-domain; asadmin-start-domain` in a PR a while back because it wasn't stopping
14:34 pameyer
but now I'm wondering if something in that change means that it doesn't stay started
14:34 pameyer
@stefankasberger welcome the the world of dataverse APIs
14:35 pameyer
@donsizemore was thinking to poke around with pids and reading asadmin; but I'm still confused
14:35 pameyer
where are you hitting it?
14:35 donsizemore
@pameyer in the past few days i've run into it in vagrant and on "bare metal"
14:36 pdurbin
stefankasberger: please feel free to request edit access and add hackathon ideas in that "projects" spreadsheet linked from the readme. Maybe we could hack on your client or something.
14:36 donsizemore
@pameyer i did a fair amount of reading about process behavior when i first cobbled dataverse-ansible together but it's been a couple years(!)
14:36 pdurbin
dataverse-ansible has entered the terrible twos
14:37 stefankasberger
@pdurbin: will talk with slava about it.
14:38 pdurbin
awesome, thanks
14:38 stefankasberger
@pameyer: thanks. all best in return. :)
14:39 pdurbin
stefankasberger: how are you installing Dataverse? With dataverse-ansible? Puppet? Chef? Salt? By hand?
14:39 donsizemore
@pdurbin first commit @donsizemore committed on Dec 1, 2015
14:40 stefankasberger
and the reason why i go to dans is to have a 2-day workshop about dataverse with other dataverse devops and devs from europe. so could be the right place to talk about this.
14:42 donsizemore
@pameyer yeah, a stop-domain isn't killing it on first run. fun
14:43 pdurbin
stefankasberger: did Slava invite Oliver?
14:43 pameyer
@donsizemore - huh. that's both bizarre and promising
14:43 donsizemore
@pdurbin @stefankasberger i keep pestering jon about odum hosting a regional dataverse dealy... seems silly with IQSS so close tho
14:44 pameyer
promising because it suggests wrapping the stop-domain in something checking the pid to wait until it's actually dead might fix it
14:44 pdurbin
donsizemore: I'd come down and hang at UNC. I assume there's decent barbeque.
14:44 pameyer
and start problem could be the stop and start stomping on each other
14:45 donsizemore
@pameyer glassfish not exiting seems to be the root problem
14:45 donsizemore
@pdurbin don't pull this vegetarian into the BBQ wars! NC BBQ has very specific traits
14:46 pdurbin
whoops
14:46 donsizemore
https://www.eater.com/2016/6/18/11966056/barbecue-sauce-styles
14:47 * pdurbin
bookmarks it
14:48 pameyer
@donsizemore that seems like a good starting point. I should be able to take a look later today (if you don't get to it first)
14:49 pameyer
... and there's a place nearby that does vegan bacon; I'm sure there's a way to make vegetarian BBQ happen ;)
14:51 donsizemore
@pameyer shrug. like a bad headache, the transient error finally went away.
14:52 donsizemore
@pameyer and the pit has wonderful mock chicken medallions http://www.thepit-raleigh.com/
14:55 donsizemore
@pdurbin i think i have 37_sampledata fixed. i'm too ashamed to tell you what the problem was.
15:06 * pdurbin
looks
15:07 pdurbin
something about burritos
15:07 pdurbin
donsizemore: want me to try it again?
15:36 pdurbin
stefankasberger: I just left this comment: https://github.com/CenterForOpenScience/osf.io/pull/8939#issuecomment-477648717
15:36 pdurbin
hope it looks ok
15:42 stefankasberger
@pdurbin: right now, i have only installed it with DataverseEU, so Docker. Will do it the normal way in the Workshop at DANS, so i understand the system in more detail. but in general i tend to use container.
15:47 juancorr93 joined #dataverse
15:48 juancorr93 left #dataverse
15:49 pdurbin
stefankasberger: you're using Slava's dataverse-docker repo?
15:52 juancorr joined #dataverse
15:55 stefankasberger
@pdurbin: so far only on my local working station, not for production. but hopefully this will be the way until the end of the year.
16:09 pdurbin
gotcha
16:51 donsizemore joined #dataverse
16:57 pdurbin
donsizemore: as of b8ea220 I'm still getting "Invalid value: <<<[]>>> for dataverseContacts"
17:03 pameyer
asadmin appears to be sensitive to either the presence of a tty or astrological alignments
17:08 donsizemore
@pameyer yis the tty bit was why my ansible scripts made manual calls until the last
17:12 pameyer
@donsizemore glad to hear it's been something you ran into; that's a sign that it might be the actual cause of the weirdness I'm seeing
17:13 pameyer
otherwise known as it being previously working by accident
17:13 donsizemore
the strange thing to me is that it seems to be transient
17:13 donsizemore
it happens in vagrant runs, sometimes 3 or 4 times in a row, then goes away. i blame... let's see, who do i blame
17:14 pameyer
I've been blaming the phase of the moon, but that seems like it's too predictable to be the actual cause
17:26 pdurbin
:)
18:40 donsizemore
@pdurbin so, vagrant: ansible-2.7.0. ec2: ansible-2.7.9
18:40 pdurbin
huh
18:40 donsizemore
@pdurbin next to drop an in-place update of ansible to see if it breaks vagrant
18:41 pdurbin
ok
18:41 donsizemore
in ec2 i'm scanning the right directory, but the expression doesn't match
18:47 pdurbin
a regular expression, I assume
18:54 donsizemore
@pdurbin it isn't the ansible version
18:54 pdurbin
bah
18:54 donsizemore
in ec2: ok: [localhost] => {"changed": false, "examined": 6, "files": [], "matched": 0, "msg": ""}
18:55 pdurbin
huh
18:55 pdurbin
so it can't find the files? can't copy them into place?
18:57 donsizemore
it's supposed to match *.sh, it's looking in the correct directory
18:57 pdurbin
and it works in vagrant
18:59 donsizemore
correct. should be simple.
18:59 pameyer
different centos7 version in ec2?
19:00 donsizemore
7.5 in vagrant, 7.6 in ec2 (i believe)
19:01 pameyer
puppetlabs/centos-7.2-64-puppet had me thinking 7.2 in vagrant; but I might be looking at the wrong branch
19:03 pdurbin
vagrant and ec2 share mostly the same code/config in dataverse-ansible itself, right? I mean, I know for ec2 I use main.yml but that there's also a vagrant.yml. So that config is different. But the playbooks are identical?
19:05 donsizemore
@pdurbin as best i know, yes. it all comes down to the pattern match in the find module...
19:06 donsizemore
(which i could just hard-code if you want to break the log-jam, but i'd rather figure out what's breaking)
19:07 donsizemore
i also noticed that call is getting single ticks, the other two use quotes. i really hope that isn't it.
19:07 pdurbin
but now you have the find module from ansible 2.7.9 on vagrant and it still works
19:13 pdurbin
yeah, you wouldn't think the single vs double quotes would matter
19:13 pdurbin
which of the 3 finds are you troubleshooting?
19:20 donsizemore
the sampleusers
19:20 donsizemore
i blame me, i'm just not sure how
19:21 donsizemore
i could just move create-users elsewhere and have it read the directory, but the idea was modularity
19:22 pdurbin
Sure. Ideally we'll be able to customize this thing.
19:25 pdurbin
donsizemore: do you want to take a break and talk about https://dataverse.org/metrics ? Or should I let you focus? :) Sorry I'm not much help with the Ansible stuff.
19:30 donsizemore
i'm tweaking the role and firing off runs in the background. the ec2 run takes quite a while
19:30 donsizemore
what's up with metrics?
19:31 pdurbin
In Firefox, if you right click that page and "show only this frame" you should be taken to https://services.dataverse.harvard.edu/miniverse/metrics/basic-viz/last12-dataverse-org?iframe=true
19:31 donsizemore
i'm still loading
19:33 pdurbin
I'm wondering if you'd be interested in running dataverse-metrics for the Dataverse community. For the 12 installations we know of that are new enough to collect metrics from. The main thing we need is for a valid SSL cert so that it works in a frame on dataverse.org without errors. The hostname doesn't matter.
19:33 pdurbin
I guess a cron job needs to be set up too.
19:33 donsizemore
sure thing. just a webserver with ssl?
19:34 donsizemore
we get free certs as long as they end in unc.edu
19:34 pdurbin
unc.edu certs are just fine
19:34 donsizemore
only 12 installations are on 4.9?
19:34 pdurbin
yeah, that I know of
19:34 pdurbin
the 12 in config.json.sample
19:35 pdurbin
but we'll need to update the list over time. I figured I'd cut new releases. v.0.2.1, v.0.2.2, etc
19:35 donsizemore
do you want a simple homepage of some sort, then the stuff beneath dataverse-metrics? my test instance got ticked about the relative path
19:35 donsizemore
i already have a box set up, all i have to do is change the hostname and request certs
19:35 donsizemore
dataversemetrics.odum.unc.edu ?
19:37 pdurbin
Let me just quick check with Danny to see if he has any thoughts on this.
19:37 donsizemore
i'm happy to do it, and can grant you shell access, just not sudo per campus policy
19:39 pdurbin
on phone, sorry
19:40 pdurbin
i probably don't need shell access
19:40 donsizemore
i know, i avoid it wherever possible ;)
19:47 donsizemore
@pdurbin i'm going to enjoy the sunshine on the way to the gym, but i'm happy to host this and only need to make minor changes to an existing VM . dataversemetrics-or-some-such.odum.unc.edu or whatever you'd like to call it and i'll have it ready tomorrow?
20:02 pdurbin
donsizemore: hi! Sorry. Danny says he's happy to have UNC host it. We're fine with whatever hostname. Thanks!
20:13 pdurbin
He even said you could add some UNC branding if you want. I just left this comment at https://github.com/IQSS/dataverse-metrics/issues/7#issuecomment-477753036
20:40 pdurbin
We could make the branding configurable. Maybe just add a line to the bottom of the page. Fed from config.json.
21:18 donsizemore joined #dataverse
23:41 bkitej joined #dataverse
