IQSS logo

IRC log for #dataverse, 2018-04-12

Connect via chat.dataverse.org to discuss Dataverse (dataverse.org, an open source web application for sharing, citing, analyzing, and preserving research data) with users and developers.

| Channels | #dataverse index | Today | | Search | Google Search | Plain-Text | plain, newest first | summary

All times shown according to UTC.

Time S Nick Message
06:48 jri joined #dataverse
06:50 jri joined #dataverse
13:49 jri joined #dataverse
13:52 jri_ joined #dataverse
14:09 donsizemore joined #dataverse
14:51 pdurbin donsizemore: thanks so much for helping with Java 7 vs Java 8 and Glassfish at https://groups.google.com/d/msg/dataverse-community/z10zmQM1YX4/U2xp9WAuBAAJ
15:05 pameyer joined #dataverse
15:20 Thali_UM joined #dataverse
15:20 Thali_UM Hi Philip
15:20 Thali_UM Good morning!
15:21 Thali_UM Who can help me with respect to the harvest of dataset?
15:21 pameyer Hi Thali_UM: I'm pretty sure pdurbin is in a meeting at the moment
15:21 pameyer he'll probably be back in a few
15:21 Thali_UM Hi pameyer
15:21 pameyer no idea if I can help or not - but what problem are you having?
15:21 Thali_UM Thank you pameyer
15:22 Thali_UM I still don't understand the part of harvesting the metadata
15:23 pameyer the way I under stand it, there are two parts to harvesting metadata.  acting as a source of metadata for another repository, or using another repository as a source of metadata for datasets to display in search
15:24 pameyer the naming of different parts of the processing in the harvesting documention tends to be confusing to me
15:24 Thali_UM there is a repository that is based on DSpace that will harvest us and ask for the protocol OAI-PMH and dublin core
15:24 Thali_UM that's why as an institution we chose Dataverse for datasets.
15:25 pameyer that sounds like you need to set up an OAI set in dataverse, and tell the details of that feed to the other repository
15:25 Thali_UM That part is what has me confused
15:26 pameyer I don't know the details about how metadata is encoded in that feed.  I know OAI-PMH is part of it, but don't know details on that spec or how it interacts w\ dublin core
15:26 pameyer ah - gotcha
15:27 Thali_UM for example I just added a metadatablock and I want to know if my values are going to be harvested.
15:27 pameyer good question - but unfortunately I don't know one way or the other
15:28 donsizemore joined #dataverse
15:28 Thali_UM It is assumed that the repository that is going to harvest us gave us the labels that go according to dublin core so that we can be harvested
15:29 pdurbin Thali_UM: hi! Data in your custom metadata block will not be harvested via the Dublin Core format. Dublic Core has basic information like "title".
15:31 pdurbin I think you know this already, but there are currently four formats: Dublin Core, DDI, JSON, and Schema.org JSON-LD. Here's a screenshot: https://i.imgur.com/xoHTEbe.png
15:33 Thali_UM Yes i know
15:34 Thali_UM Then how can my metadata be harvested?
15:34 pdurbin Can you put your metadata in Dublin Core?
15:35 Thali_UM It is assumed that the repository that is going to harvest us, gave us the labels that dublin core respects so that we can be harvested.
15:36 Thali_UM I don't understand your question?
15:39 pdurbin Your DSpace installatoin wants to harvest from your Dataverse installation (OAI-PMH) using the Dublin Core format, right?
15:40 Thali_UM yes, DSpace will harvest Dataverse
15:41 Thali_UM but Dspace is another institution, we do not know how they handle it, but they have to harvest us through the OAI-PMH and the use of dublin core
15:42 pdurbin Thali_UM: here's the Dataverse code that produces the Dublin Core format: https://github.com/IQSS/dataverse/blob/v4.8.6/src/main/java/edu/harvard/iq/dataverse/export/dublincore/DublinCoreExportUtil.java#L97
15:44 pdurbin In that code you'll see fields like "title" and "description", etc. Those are the fields you should use if you want the data to be harvested using the Dublin Core format.
15:48 pdurbin Does that make sense?
15:50 Thali_UM I just shared an image in your email
15:51 Thali_UM Yes, a little more sense
15:52 pdurbin Good. :)
15:52 Thali_UM What are the instructions to modify the code? Download the project, load it in Netbeans, compress it and use that package to install Dataverse?
15:53 pdurbin Thali_UM: I just updated the instructions. Here they are: http://guides.dataverse.org/en/4.8.6/developers/dev-environment.html
15:55 pdurbin Thali_UM: I got the image you sent via email but can you please send it to support@dataverse.org to create a ticket? (And please also summarize what you are trying to do, of course.)
15:56 Thali_UM Thank you Philip :D
15:56 Thali_UM Yes
15:57 Thali_UM Does that make sense, with the image?
15:58 pdurbin The image helps.
15:59 pdurbin Thali_UM: once you have a ticket number, please post it here so anyone reading this in the IRC log can find it.
15:59 Thali_UM yes
16:00 pdurbin thanks!
16:01 pdurbin andrewSC bjonnh bricas donsizemore knikolla pameyer Thali_UM check this out. The students I'm mentoring posted a video showing the high-availability setup for PostgreSQL within Docker/Kubernetes/OpenShift they are working on: https://groups.google.com/d/msg/dataverse-community/TSxf4MTYYjg/7VJB_-GJBAAJ
16:02 pameyer pdurbin: I saw
16:02 pameyer took a quick look to see where the postgres replication was hiding in that repo
16:03 pameyer didn't find it, but may check back later
16:09 Thali_UM #260805.
16:10 andrewSC https://media.giphy.com/media/l46CkATpdyLwLI7vi/giphy.gif
16:10 andrewSC nice!!!
16:18 pdurbin pameyer: I believe the postgres replication magic is hiding in here: https://github.com/sclorg/postgresql-container/blob/c99a552076295476dfcd2ab809ea9dcff036ac7d/examples/replica/README.md
16:19 pdurbin Thali_UM: thanks
16:19 pdurbin andrewSC: you like? :)
16:20 donsizemore @pdurbin have you checked out CockroachDB?
16:21 andrewSC very much so, I'm actually in the process of introducing CI/CD to the lab down here and we're staring squarely at kubernetes and docker for how we'll be managing and deploying services going forward
16:21 andrewSC all good stuff
16:24 pdurbin donsizemore: I heard about CockroachDB at https://changelog.com/gotime/73 but that's about it.
16:25 pdurbin andrewSC: speaking of CI/CD, check out how there's a new "build passing" (with Travis) button on donsizemore's https://github.com/IQSS/dataverse-ansible
16:25 pameyer is CockroachDB a DB or a CI/CD tool?
16:26 pameyer joined #dataverse
16:26 donsizemore @pameyer it's postgres-compatible (for the most part) and designed to survive a newkular disaster
16:26 pdurbin And pameyer reminded me (thanks) that if donsizemore's ears were burning during the community call, it's because people were asking about dataverse-ansible: https://groups.google.com/d/msg/dataverse-community/fT98c0eGgCc/9rBkIYmuAQAJ
16:26 pameyer @donsizemore - thanks
16:27 pdurbin Which reminds me that I should follow up on https://github.com/IQSS/dataverse-ansible/issues/9 since she's the one who was asking.
16:28 pameyer and the general conversations on dataverse-ansible reminds me that I should probably try that one out, and see if I could switch to it (vs the one I'm using now)
16:28 pameyer which probably won't happen this week, or maybe this month :(
16:31 donsizemore @pdurbin the role wasn't written to be idempotent
16:31 pdurbin Ok, I just left a comment on that issue (9).
16:31 pdurbin donsizemore: oh. Is that bad? It sounds bad. :)
16:32 pameyer yeah - idempotent dv install / config can be tricky sometimes
16:38 pdurbin yeah
16:38 donsizemore @pameyer well, ansible may aim for idempotence but glassfish certainly doesn't, so i didn't bother.
16:39 pdurbin Is payara any better in that regard?
16:41 donsizemore s/glassfish/dataverse/ (install.pl for instance declines to proceed if the database exists). we could make the role idempotent though
16:42 pdurbin Ok, makes sense. It's not Glassfish's fault. It's the Dataverse installer.
16:43 donsizemore @pdurbin well, nobody's fault, it's just... historically been a one-shot process
16:43 donsizemore @pdurbin and that can be changed with semaphores
16:44 pdurbin yeah, about the one shot
16:45 pdurbin not sure what you mean by semaphores
16:53 donsizemore @pdurbin if one can't elegantly detect say whether the database tables had been created (or something less elegantly detected, IDK) you could touch say /etc/ansible/dataverse_db_created then test for that on subsequent runs
16:54 pdurbin Hmm. I'm sure you're right but my head is in a different space right now. I'm working on https://github.com/IQSS/dataverse/issues/1916
17:11 xyz1 joined #dataverse
17:21 jri joined #dataverse
17:32 pameyer @donsizemore: I did a bunch of scripts creating output files for wrapping various asadmin / dataverse config options.  was an ugly hack, but mostly worked
17:53 pameyer left #dataverse
19:15 pameyer joined #dataverse
21:47 jri joined #dataverse
21:56 donsizemore joined #dataverse
22:17 soU joined #dataverse
22:48 pameyer left #dataverse
23:28 hypraware10HC7H joined #dataverse

| Channels | #dataverse index | Today | | Search | Google Search | Plain-Text | plain, newest first | summary

Connect via chat.dataverse.org to discuss Dataverse (dataverse.org, an open source web application for sharing, citing, analyzing, and preserving research data) with users and developers.