Time
S
Nick
Message
06:48 jri joined #dataverse
06:50 jri joined #dataverse
13:49 jri joined #dataverse
13:52 jri_ joined #dataverse
14:09 donsizemore joined #dataverse
14:51 pdurbin
donsizemore: thanks so much for helping with Java 7 vs Java 8 and Glassfish at https://groups.google.com/d/msg/dataverse-community/z10zmQM1YX4/U2xp9WAuBAAJ
15:05 pameyer joined #dataverse
15:20 Thali_UM joined #dataverse
15:20 Thali_UM
Hi Philip
15:20 Thali_UM
Good morning!
15:21 Thali_UM
Who can help me with respect to the harvest of dataset?
15:21 pameyer
Hi Thali_UM: I'm pretty sure pdurbin is in a meeting at the moment
15:21 pameyer
he'll probably be back in a few
15:21 Thali_UM
Hi pameyer
15:21 pameyer
no idea if I can help or not - but what problem are you having?
15:21 Thali_UM
Thank you pameyer
15:22 Thali_UM
I still don't understand the part of harvesting the metadata
15:23 pameyer
the way I under stand it, there are two parts to harvesting metadata. acting as a source of metadata for another repository, or using another repository as a source of metadata for datasets to display in search
15:24 pameyer
the naming of different parts of the processing in the harvesting documention tends to be confusing to me
15:24 Thali_UM
there is a repository that is based on DSpace that will harvest us and ask for the protocol OAI-PMH and dublin core
15:24 Thali_UM
that's why as an institution we chose Dataverse for datasets.
15:25 pameyer
that sounds like you need to set up an OAI set in dataverse, and tell the details of that feed to the other repository
15:25 Thali_UM
That part is what has me confused
15:26 pameyer
I don't know the details about how metadata is encoded in that feed. I know OAI-PMH is part of it, but don't know details on that spec or how it interacts w\ dublin core
15:26 pameyer
ah - gotcha
15:27 Thali_UM
for example I just added a metadatablock and I want to know if my values are going to be harvested.
15:27 pameyer
good question - but unfortunately I don't know one way or the other
15:28 donsizemore joined #dataverse
15:28 Thali_UM
It is assumed that the repository that is going to harvest us gave us the labels that go according to dublin core so that we can be harvested
15:29 pdurbin
Thali_UM: hi! Data in your custom metadata block will not be harvested via the Dublin Core format. Dublic Core has basic information like "title".
15:31 pdurbin
I think you know this already, but there are currently four formats: Dublin Core, DDI, JSON , and Schema.org JSON-LD. Here's a screenshot: https://i.imgur.com/xoHTEbe.png
15:33 Thali_UM
Yes i know
15:34 Thali_UM
Then how can my metadata be harvested?
15:34 pdurbin
Can you put your metadata in Dublin Core?
15:35 Thali_UM
It is assumed that the repository that is going to harvest us, gave us the labels that dublin core respects so that we can be harvested.
15:36 Thali_UM
I don't understand your question?
15:39 pdurbin
Your DSpace installatoin wants to harvest from your Dataverse installation (OAI-PMH) using the Dublin Core format, right?
15:40 Thali_UM
yes, DSpace will harvest Dataverse
15:41 Thali_UM
but Dspace is another institution, we do not know how they handle it, but they have to harvest us through the OAI-PMH and the use of dublin core
15:42 pdurbin
Thali_UM: here's the Dataverse code that produces the Dublin Core format: https://github.com/IQSS/dataverse/blob/v4.8.6/src/main/java/edu/harvard/iq/dataverse/export/dublincore/DublinCoreExportUtil.java#L97
15:44 pdurbin
In that code you'll see fields like "title" and "description", etc. Those are the fields you should use if you want the data to be harvested using the Dublin Core format.
15:48 pdurbin
Does that make sense?
15:50 Thali_UM
I just shared an image in your email
15:51 Thali_UM
Yes, a little more sense
15:52 pdurbin
Good. :)
15:52 Thali_UM
What are the instructions to modify the code? Download the project, load it in Netbeans, compress it and use that package to install Dataverse?
15:53 pdurbin
Thali_UM: I just updated the instructions. Here they are: http://guides.dataverse.org/en/4.8.6/developers/dev-environment.html
15:55 pdurbin
Thali_UM: I got the image you sent via email but can you please send it to support
15:56 Thali_UM
Thank you Philip :D
15:56 Thali_UM
Yes
15:57 Thali_UM
Does that make sense, with the image?
15:58 pdurbin
The image helps.
15:59 pdurbin
Thali_UM: once you have a ticket number, please post it here so anyone reading this in the IRC log can find it.
15:59 Thali_UM
yes
16:00 pdurbin
thanks!
16:01 pdurbin
andrewSC bjonnh bricas donsizemore knikolla pameyer Thali_UM check this out. The students I'm mentoring posted a video showing the high-availability setup for PostgreSQL within Docker/Kubernetes/OpenShift they are working on: https://groups.google.com/d/msg/dataverse-community/TSxf4MTYYjg/7VJB_-GJBAAJ
16:02 pameyer
pdurbin: I saw
16:02 pameyer
took a quick look to see where the postgres replication was hiding in that repo
16:03 pameyer
didn't find it, but may check back later
16:09 Thali_UM
#260805.
16:10 andrewSC
https://media.giphy.com/media/l46CkATpdyLwLI7vi/giphy.gif
16:10 andrewSC
nice!!!
16:18 pdurbin
pameyer: I believe the postgres replication magic is hiding in here: https://github.com/sclorg/postgresql-container/blob/c99a552076295476dfcd2ab809ea9dcff036ac7d/examples/replica/README.md
16:19 pdurbin
Thali_UM: thanks
16:19 pdurbin
andrewSC: you like? :)
16:20 donsizemore
@pdurbin have you checked out CockroachDB?
16:21 andrewSC
very much so, I'm actually in the process of introducing CI/CD to the lab down here and we're staring squarely at kubernetes and docker for how we'll be managing and deploying services going forward
16:21 andrewSC
all good stuff
16:24 pdurbin
donsizemore: I heard about CockroachDB at https://changelog.com/gotime/73 but that's about it.
16:25 pdurbin
andrewSC: speaking of CI/CD, check out how there's a new "build passing" (with Travis) button on donsizemore's https://github.com/IQSS/dataverse-ansible
16:25 pameyer
is CockroachDB a DB or a CI/CD tool?
16:26 pameyer joined #dataverse
16:26 donsizemore
@pameyer it's postgres-compatible (for the most part) and designed to survive a newkular disaster
16:26 pdurbin
And pameyer reminded me (thanks) that if donsizemore's ears were burning during the community call, it's because people were asking about dataverse-ansible: https://groups.google.com/d/msg/dataverse-community/fT98c0eGgCc/9rBkIYmuAQAJ
16:26 pameyer
@donsizemore - thanks
16:27 pdurbin
Which reminds me that I should follow up on https://github.com/IQSS/dataverse-ansible/issues/9 since she's the one who was asking.
16:28 pameyer
and the general conversations on dataverse-ansible reminds me that I should probably try that one out, and see if I could switch to it (vs the one I'm using now)
16:28 pameyer
which probably won't happen this week, or maybe this month :(
16:31 donsizemore
@pdurbin the role wasn't written to be idempotent
16:31 pdurbin
Ok, I just left a comment on that issue (9).
16:31 pdurbin
donsizemore: oh. Is that bad? It sounds bad. :)
16:32 pameyer
yeah - idempotent dv install / config can be tricky sometimes
16:38 pdurbin
yeah
16:38 donsizemore
@pameyer well, ansible may aim for idempotence but glassfish certainly doesn't, so i didn't bother.
16:39 pdurbin
Is payara any better in that regard?
16:41 donsizemore
s/glassfish/dataverse/ (install.pl for instance declines to proceed if the database exists). we could make the role idempotent though
16:42 pdurbin
Ok, makes sense. It's not Glassfish's fault. It's the Dataverse installer.
16:43 donsizemore
@pdurbin well, nobody's fault, it's just... historically been a one-shot process
16:43 donsizemore
@pdurbin and that can be changed with semaphores
16:44 pdurbin
yeah, about the one shot
16:45 pdurbin
not sure what you mean by semaphores
16:53 donsizemore
@pdurbin if one can't elegantly detect say whether the database tables had been created (or something less elegantly detected, IDK) you could touch say /etc/ansible/dataverse_db_created then test for that on subsequent runs
16:54 pdurbin
Hmm. I'm sure you're right but my head is in a different space right now. I'm working on https://github.com/IQSS/dataverse/issues/1916
17:11 xyz1 joined #dataverse
17:21 jri joined #dataverse
17:32 pameyer
@donsizemore: I did a bunch of scripts creating output files for wrapping various asadmin / dataverse config options. was an ugly hack, but mostly worked
17:53 pameyer left #dataverse
19:15 pameyer joined #dataverse
21:47 jri joined #dataverse
21:56 donsizemore joined #dataverse
22:17 soU joined #dataverse
22:48 pameyer left #dataverse
23:28 hypraware10HC7H joined #dataverse
