IQSS logo

IRC log for #dataverse, 2019-04-10

Connect via chat.dataverse.org to discuss Dataverse (dataverse.org, an open source web application for sharing, citing, analyzing, and preserving research data) with users and developers.

| Channels | #dataverse index | Today | | Search | Google Search | Plain-Text | plain, newest first | summary

All times shown according to UTC.

Time S Nick Message
06:53 juancorr joined #dataverse
06:55 poikilotherm joined #dataverse
07:06 jri joined #dataverse
07:56 MrK joined #dataverse
10:08 pdurbin Hi, all. If you're blocked or otherwise need anything, please let me know.
10:22 poikilotherm pdurbin do you have responses from upgrades to 4.12 and strange data loses?
10:22 poikilotherm I have no clue how it happened, but at some point yesterday I lost our test instance
10:22 poikilotherm Hard to say if that happened before or after I upgraded
10:30 pdurbin Lost your test instance? What does that mean? Is that like terminating an EC2 instance?
10:30 poikilotherm I lost all test data in it, configs like ORCID login etc
10:30 poikilotherm Just like a database drop
10:31 poikilotherm But I did no bootstrap afterwards
10:31 poikilotherm Its totally bezare
10:31 poikilotherm bizare
10:34 poikilotherm The index etc is still present
10:55 pdurbin I can only assume you're using dataverse-kubernetes rather than our old boring ways. :)
11:08 jri_ joined #dataverse
11:14 jri joined #dataverse
11:19 poikilotherm Yes! :-D
11:21 pdurbin I'm feeling a little blocked to help. Blocked by https://github.com/IQSS/dataverse-kubernetes/issues/12
11:21 poikilotherm :-D
11:26 pdurbin Nice. DataverseEU meeting happening now: https://twitter.com/4tykhonov/status/1115915691444592641
11:52 donsizemore joined #dataverse
12:19 pdurbin donsizemore: morning. I've been thinking. I can probably just write a little script on the side to put my key and cert into place.
12:31 donsizemore @pdurbin you're welcome to do that, but i promise the https/proxy stuff is on my list
12:49 pdurbin donsizemore: cool. Can I ask a favor. Related to Jenkins.
13:28 donsizemore graphs?
13:29 pdurbin Heh. No, I was wondering if you could set https://github.com/IQSS/dataverse/issues/5725 into motion will a pull request. I'm happy to pile on some commits.
13:35 donsizemore Jenkins, Travis, what else?
13:36 pdurbin Buh. I think that's it. poikilotherm is using gitlab-ci but I don't think it's a resource for the community.
13:50 donsizemore there's a handy-dandy gitlab-ce rpm, but i haven't done much with it
13:55 pdurbin ce?
14:00 poikilotherm community edition
14:00 pdurbin ah
14:01 poikilotherm You don't need to install your own GitLab CE
14:01 poikilotherm You can just use Gitlab.com
14:01 poikilotherm Its free and has GitLab CI attached to it
14:01 pdurbin wow, nice
14:01 pdurbin so free like Travis
14:02 pdurbin donsizemore: by the way I *love* that you wrote "in the near future will deploy it to a test VM" at https://github.com/IQSS/dataverse-jenkins
14:03 donsizemore it's nice to have things to look forward to =)
14:03 pdurbin poikilotherm: if you were going to contribute a job definition (is that what they're called?) to that dataverse-jenkins repo, what would you call the file? dataverse-kubernetes.xml?
14:03 poikilotherm https://about.gitlab.com/features/#verify
14:03 poikilotherm Its a yaml file
14:04 poikilotherm Maybe it should live in a subfolder called gitlab
14:04 poikilotherm And normally it is called .gitlab-ci.yml
14:04 pdurbin poikilotherm: sorry, I'm talking about Jenkins. I think Jenkins uses XML (or JSON?) rather than YAML.
14:04 poikilotherm Whoops
14:04 poikilotherm I thought you were talking about Gitlab
14:05 pdurbin I'm all over the place. :)
14:05 poikilotherm Depends
14:05 poikilotherm Just yesterday I found https://medium.com/@gustavo.guss/jenkins-building-docker-image-and-sending-to-registry-64b84ea45ee9
14:05 donsizemore @pdurbin should we rename it dataverse-ci?
14:05 poikilotherm That pipeline file doesn't look like XML at all ;-)
14:06 pdurbin No, I still think we should focus on Jenkins for now.
14:06 pdurbin I can make a dataverse-gitlab-ci for poikilotherm if he wants.
14:07 pdurbin repo, that is, under IQSS
14:07 poikilotherm Overkill.
14:08 pdurbin poikilotherm: you can just throw your .gitlab-ci.yml into your dataverse-kubernets repo if you haven't already.
14:08 poikilotherm Aye
14:09 pdurbin What do people think about my monitoring idea? Please see https://groups.google.com/d/msg/dataverse-community/zziMWREdirs/8TPfzSxeBQAJ
14:11 donsizemore @pdurbin naggy-ose? monit? diddukewin.com?
14:11 pdurbin I've heard of the first two. :)
14:12 donsizemore the 3rd was a (half-) joke
14:12 pdurbin ah :)
14:12 donsizemore it's a very useful website during the NCAA tournament
14:13 poikilotherm pdurbin maybe you are looking for sth like https://cachethq.io/
14:14 pdurbin ah, you should see Sherry and Courtney in the notes from the call yesterday: https://groups.google.com/d/msg/dataverse-community/JHIfe2uqvkc/GjLbOfIZAwAJ #marchmadness
14:14 pdurbin poikilotherm: well, I certainly don't want pages/alerts
14:14 pdurbin I'm thinking of something like https://status.datacite.org
14:15 pdurbin "We continuously monitor the status of datacite.org and all its related services."
14:15 pdurbin related services being the operative term
14:16 poikilotherm Yeah. That page is based on statuspage.io
14:16 poikilotherm cachethq.io is sth similar to that
14:16 poikilotherm But OSS
14:17 pdurbin oh, interesting
14:17 * pdurbin looks closer
14:27 donsizemore @pdurbin for the dev guide, should i stick 'continuous integration' under the testing heading, or should it have its own heading?
14:28 pdurbin well, there's a whole page on testing, not just a heading
14:28 donsizemore does CI need its own page, or should it pop inside testing
14:29 pdurbin ah, ah, in the testing page is fine... I'm trying to think where
14:29 donsizemore this is why i'm asking =)
14:29 pdurbin maybe above "The Phoenix Server" for now
14:29 pdurbin I'll hack on it once you set it in motion.
14:34 donsizemore 10-4
14:34 pdurbin thanks much
14:41 donsizemore joined #dataverse
15:12 pdurbin donsizemore: I'm about to go to standup but when you have a moment, I'm hoping to pick your brain about https://github.com/IQSS/dataverse/issues/5722
15:20 larsks joined #dataverse
15:21 larsks Anyone around? I am trying to help someone working on dataverse diagnose an auth problem, but it's been a couple of decades since I had to build java code...
15:25 larsks pdurbin: ^^^^ :)
15:27 pdurbin larsks: shib? oauth? "builtin"?
15:27 pdurbin nice to see you here :)
15:29 larsks pdurbin: actually "joss". kevin condon is going something with swift.  It's hitting keystone apparently @ https://github.com/javaswift/joss/blob/master/src/main/java/org/javaswift/joss/command/shared/identity/access/KeystoneV3Access.java and it's getting back a response that doesn't have a catalog. I strongly suspect user configuration error, but I would feel better if I could run some test code myself.
15:30 pdurbin Funny. I just heard at standup "we got the joss guys' attention." :)
15:30 pdurbin I thought you meant users logging into Dataverse. You're talking about Dataverse auth'ing to a storage driver.
15:31 larsks Sure.
15:31 larsks What I want to do is put together some sort of three-line "authenticate to keystone, list containers" example that I can try out locally.
15:31 larsks Ideally that reads auth configuration the same way as the actual code...
15:31 pdurbin in java
15:32 larsks Yes. I can write Python until I'm blue in the face :)
15:32 donsizemore @pdurbin I noticed in .travis.yml you're calling maven with "-DcompilerArgument=-Xlint:unchecked test -P all-unit-tests" — want those options in jenkins?
15:32 pdurbin I know, I remember the kata you added for me. Thanks again. :)
15:34 pdurbin donsizemore: yes, please
15:34 pdurbin larsks: would it make sense for you to add an issue to https://github.com/javaswift/joss/issues ?
15:34 larsks I don't think so: I'm not convinced that there is a joss issue yet.
15:35 larsks That's why I would like to put together some test code to either reproduce the problem (or not).
15:35 larsks There is a joss tutorial @ https://github.com/javaswift/tutorial-joss-quickstart which seems to build correctly, but I have no idea how to "make it go".
15:36 larsks Plus the tutorial is super old.
15:36 pdurbin larsks: ok, could you create an issue over at https://github.com/IQSS/dataverse/issues then?
15:38 donsizemore @pdurbin done.
15:42 larsks pdurbin: https://github.com/IQSS/dataverse/issues/5750
15:42 pdurbin larsks: thanks!
15:43 pdurbin donsizemore: thanks!
15:45 pdurbin larsks: I just left this comment: https://github.com/IQSS/dataverse/issues/5478#issuecomment-481746253
15:46 pdurbin larsks: but I'm thinking... what if I spin you up an installation of Dataverse on EC2 and share the ssh key with you so we can configure Dataverse for Swift and troubleshoot a bit.
15:47 larsks Sure, worth a shot.
15:47 pdurbin Should I send the key to the email on your GitHub profile?
15:48 larsks Sure. Or you can grab my public keys from https://github.com/larsks.keys and just drop those in place.
15:48 larsks I favor the latter method, since it doesn't involve emailing secrets :)
15:48 pdurbin sure, good idea
15:49 pdurbin donsizemore: man, that spin up script comes in handy. Thanks again. :)
15:56 pdurbin TASK [dataverse : start glassfish with systemd] ******
15:56 pdurbin almost there, I hope
15:59 pdurbin larsks: ok, please try this: ssh centos@ec2-54-208-127-237.compute-1.amazonaws.com
15:59 donsizemore @pdurbin pile on, pile on https://github.com/IQSS/dataverse/pull/5751
15:59 pdurbin here's the server: http://ec2-54-208-127-237.compute-1.amazonaws.com:8080
16:00 pdurbin donsizemore: thanks! Also, why didn't we think of just adding ssh keys from github? :)
16:00 pdurbin larsks: how about I stub out a file for swift creds and you fill them in
16:01 larsks pdurbin: sure, that sounds great.
16:03 pdurbin larsks: this is what I'm following: http://guides.dataverse.org/en/4.12/installation/config.html#swift-storage
16:04 pdurbin larsks: ok, please edit /usr/local/glassfish4/glassfish/doma​ins/domain1/config/swift.properties
16:04 donsizemore @pdurbin i'm trying not to think about it =)
16:04 pdurbin donsizemore: I added my key too. Why not? :)
16:05 larsks pdurbin: thanks, logged in.
16:06 larsks pdurbin: do I need to provide swift.swift_endpoint.endpoint1? Shouldn't that come from the catalog?
16:06 pdurbin I don't know. Sorry.
16:07 larsks Okay.  I've filled in swift.properties. What's next?
16:08 pdurbin I just ran this: ./asadmin $ASADMIN_OPTS create-jvm-options "\-Ddataverse.files.storage-driver-id=swift"
16:08 larsks Okay. So...what do I do?
16:08 pdurbin uh, are we ok with "_" (underscore) as the "custom container name separator"?
16:09 larsks I...uh, I guess so? I'm just going to delete anything that gets created.  There are probably pretty low quotas on my account right now, so if this is going to try creating tons of stuff it might just fail for that reason.
16:09 larsks Although maybe not for object storage... I guess we'll see.
16:11 pdurbin Ok, I think I'm done configuring Swift. Why don't you tail /usr/local/glassfish4/glassfish/​domains/domain1/logs/server.log
16:11 larsks Tailing, aye.
16:12 pdurbin don't make fun of our chatty logs... we know, we know: https://github.com/IQSS/dataverse/issues/5749
16:12 larsks pdurbin: I always thought that was the defining characteristic of a java application :)
16:13 pdurbin sigh :)
16:14 pdurbin Ok, I just uploaded and published this file: http://ec2-54-208-127-237.compute-1.amazonaws.com:8080/file.xhtml?fileId=19 (50by1000.tab)
16:14 larsks That means it worked, right?
16:14 pdurbin It's in http://ec2-54-208-127-237.compute-1.amazonaws.com:8080/dataset.xhtml?persistentId=doi:10.5072/FK2/0KUQ4X so if it's on disk, I'll be in a directory called 0KUQ4X on the server. But I hope it isn't. I hope its in your Swift. Is it?
16:15 larsks It doesn't appear to be.  There are no swift containers associated with my account other than some test ones that have been floating around there for a while.
16:15 pdurbin Yeah, it's on disk. `file` shows /usr/local/dvn/data/10.5072/FK2/0KU​Q4X/16a0805e53d-a66af0c660fd.orig: data
16:16 pdurbin even in the future nothing works
16:18 larsks if you take a look at that github issue, I've spotted the root cause of the problem.
16:18 larsks But I don't know if the solution is configuration or code.
16:18 pdurbin You have "auth_type" set to "password"
16:18 larsks I do.
16:18 pdurbin "auth_type can either be keystone, keystone_v3, or it will assumed to be basic." (from the dataverse installation guide linked above)
16:18 larsks Ah, see, I didn't know what it was expecting :)
16:18 larsks Let me update that.
16:19 larsks Now it is keystone_v3.
16:19 pdurbin ok, I'll make another dataset
16:20 larsks pdurbin: can we get http logs out of this somehow?
16:20 pdurbin well, server.log is the main log for glassfish
16:20 larsks Sure, but I mean, can we get a log of the keystone request and response?
16:21 larsks Kevin generated that somehow, but I'm sure that doesn't happen by default.
16:21 larsks That would be chatty, even for a java app.
16:21 pdurbin /usr/local/dvn/data/10.5072/FK2/​3YLPEL/16a080d51e7-1962f6e1fd69: JPEG image data, EXIF standard
16:22 larsks Hmm, still nothing in swift.
16:22 pdurbin Yeah. 3YLPEL is the directory for the second dataset. The file I uploaded is on disk.
16:23 pdurbin I wonder if I have to restart Glassfish.
16:23 pdurbin this shows swift: grep swift /usr/local/glassfish4/glassfish/d​omains/domain1/config/domain.xml
16:23 pdurbin let me do that
16:24 larsks I will brb.
16:25 pdurbin ooo.  Error – Failed to add files to the dataset. Please try uploading the file(s) again. If you believe this is an error, please contact Root Support for assistance.
16:26 * larsks is back
16:26 pdurbin Command exception, HTTP Status code: 300 => UNKNOWN
16:26 pdurbin at org.javaswift.joss.command.impl.cor​e.httpstatus.HttpStatusChecker.veri​fyCode(HttpStatusChecker.java:45)
16:27 pdurbin Failed to save the file, storage id swift://endpoint1:doi-10-5072:FK2​/1PGGN7/16a0811be1e-0f1a9b4eb33d (SwiftAccessIO: failed to authenticate keystone_v3 for the end point endpoint1)|#]
16:29 * larsks is mystified. Those credentials all look correct.
16:30 pdurbin We're using joss 0.10.0 if that helps: https://github.com/IQSS/dataverse/blob/v4.12/pom.xml#L477
16:31 larsks You got me.  At this point I would look to a dataverse/joss person, since from the openstack perspective all those credentials look fine.
16:31 larsks Or if we could get an http transaction log to see what request is actually being sent over...
16:32 pdurbin let me post the stack trace and screenshot. I see you're talking to Leonid in the issue you opened.
16:33 larsks Yeah. If leonid/kevin are able to try out that config change, it may turn out everything Just Works.
16:34 larsks But I feel like we may have lost them to lunch or something.
16:35 pdurbin I'm heading to lunch myself but I just posted the stacktrace and will swing by their office.
16:35 larsks Awesome. I think I will also find some food.
16:46 pdurbin Here's the line from the stack trace: https://github.com/IQSS/dataverse/blob/v4.12/src/main/java/edu/harvard/iq/dataverse/dataaccess/SwiftAccessIO.java#L794
19:10 andrewSC joined #dataverse
19:34 donsizemore joined #dataverse
20:02 donsizemore @pdurbin i'm about to head out but i can test 5722 in vagrant tomorrow, and ec2 is well?
20:17 pdurbin ah, I didn't even think of testing it in vagrant

| Channels | #dataverse index | Today | | Search | Google Search | Plain-Text | plain, newest first | summary

Connect via chat.dataverse.org to discuss Dataverse (dataverse.org, an open source web application for sharing, citing, analyzing, and preserving research data) with users and developers.