Time
S
Nick
Message
07:05 juancorr joined #dataverse
07:07 jri joined #dataverse
09:14 jri joined #dataverse
09:17 jri_ joined #dataverse
09:18 dataverse-user joined #dataverse
09:31 dataverse-user
Hi,I was wondering how much effort it would take to create another the publish-like functionality in dataverse that only creates a new version but keeps this version not being visible to everyone so that people can modify and work on a dataset until at some time they finally publish it.
10:06 pdurbin joined #dataverse
10:06 pdurbin
dataverse-user: hi! Still there? My first thought it to try the Private URL feature.
10:32 dataverse-user joined #dataverse
10:35 dataverse-user42 joined #dataverse
10:36 dataverse-user joined #dataverse
10:41 dataverse-user
Hi @pdurbin. Thanks for your suggestion. Unfortunately this does not allow to create multiple versions but only using the latest draft. I would wish to have persistent URIs for multiple versions that can be triggered (as currently by the publish-button) but for private urls/some logged in users. I was wondering if this might be able by manually modifying some settings in the database?
10:53 pdurbin
dataverse-user: ok, have you played around with restricting files?
10:53 pdurbin
candy`: good morning to you too :)
11:11 poikilotherm joined #dataverse
11:24 dataverse-user joined #dataverse
11:30 dataverse-user
@pdurbin. I am not yet an expert in using dataverse. Do you mean [1]? I did not really know what effect this setting has, as I think the files are restricted per default. [1] http://guides.dataverse.org/en/latest/user/dataset-management.html
11:55 candy`
pdurbin: :)
12:04 donsizemore joined #dataverse
12:05 donsizemore
@pdurbin hey hey, in updating dataverse-ansible to 4.9.3 i found that the upgraded ansible provisioner in vagrant broke conditionals. it was friday evening, so i didn't iron that out. it's on my list for today.
12:12 pdurbin_m joined #dataverse
12:13 pdurbin_m
donsizemore: thanks!
12:14 donsizemore
@pdurbin_m don't thank me until it works. in reading more about conditionals in 2.x, i don't see how it worked in the first place
12:29 donsizemore
@pdurbin_m p.s. i'll add the commit as well
12:53 pdurbin
donsizemore: well, if I can help, please let me know
12:58 jri joined #dataverse
13:08 donsizemore
@pdurbin is there a helper script (or other page in the developer guide) i can follow to see what commit variable you want me to grab?
13:09 donsizemore
n/m, i see BRANCH_COMMIT with that urgly git one-liner
13:10 pdurbin
BRANCH_COMMIT=$(git rev-parse --abbrev-ref HEAD)-$(git log --oneline | head -1 | awk '{print $1}')
13:12 pdurbin
You already have the branch so really you only need the `git log --oneline` part
13:17 poikilotherm
Git log-compact for the win... https://mackyle.github.io/git-log-compact/
13:17 poikilotherm
And you guys know that you can create shortcut commands yourself, right?
13:18 poikilotherm
Just put an alias in the git config... ;-)
13:18 pdurbin
yes, I have aliases in https://github.com/pdurbin/dotfiles/blob/master/.gitconfig
13:18 pdurbin
but we're talking about this script: https://github.com/IQSS/dataverse/blob/v4.9.3/scripts/installer/custom-build-number
13:28 poikilotherm
Ah ok - that's a different story... ;-)
13:38 pdurbin
poikilotherm: I'm seeing very strange JUnit behavior after merging the latest from "develop" into my 4261-sitemap branch. I'm concerned the JUnit upgrade is breaking something.
13:44 poikilotherm
Could you be a bit more specific? Can I check out the branch and build it to see the errors?
13:45 poikilotherm
Willing to help :-)
13:52 poikilotherm
It's
13:52 poikilotherm
It's #5084, right?
13:54 pdurbin
poikilotherm: yes, please take a look at this issue I just opened: https://github.com/IQSS/dataverse/issues/5122
14:04 djbrooke joined #dataverse
14:05 poikilotherm
pdurbin: just commented on the issue :-)
14:05 poikilotherm
We can talk here, but I'll be picking up kids in about 40 minutes
14:06 donsizemore
@pdurbin oy vey build.number=5027-dashboard-admin-guide, build {'stderr_lines': [], u'changed': True, u'end': u'2018-10-01 16:03:47.214710', 'failed': False, u'stdout': u'', u'cmd': [u'cd', u'/tmp/dataverse', u'&&', u'git', u'rev-parse', u'--abbrev-ref', u'HEAD)-$(git', u'log', u'--oneline', u'|', u'head', u'-1', u'|', u'awk', u'{print $1}'], u'rc': 0, u'start': u'2018-10-01 16:03:47.208258', u'stderr': u'', u'delta': u'0:00:00.006452',
14:10 pdurbin
donsizemore: having trouble? I'm not sure what that output means.
14:11 donsizemore
@pdurbin it means sometimes ansible makes me feel like i'm standing in at the Nightmare Before Christmas toy production line
14:11 pdurbin
heh
14:11 * pdurbin
sings "this is halloween, this is halloween"
14:12 donsizemore
@pdurbin if i can get it to register just the commit i think we're good on all counts
14:12 pameyer joined #dataverse
14:12 pdurbin
sure, again, you've already got the branch in there
14:18 poikilotherm
pdurbin: installing Netbeans 8.2 ATM
14:18 pdurbin
poikilotherm: thank you!
14:18 poikilotherm
you're welcome ;-)
14:23 poikilotherm
Netbeans seems to escape the $argLine argument that is used for JaCoCo
14:23 poikilotherm
"Command was /bin/sh -c cd /home/obertuch/Entwicklung/dataverse && /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.181.b15-0.fc28.x86_64/jre/bin/java '${argLine}' -Duser.timezone=UTC "
14:24 poikilotherm
This is not working on cmd, too, because of the "'" around the var.
14:25 poikilotherm
Why are you guys using that anyway? Normally JaCoCo plugin is taking care of adding the agent itself...
14:26 pdurbin
poikilotherm: you added that argLine in https://github.com/IQSS/dataverse/pull/5061
14:27 poikilotherm
Yes, but I was just re-adding it
14:27 poikilotherm
I accidentaly removed it before
14:27 poikilotherm
Gimme a sec, I will look it up :-D
14:29 djbrooke joined #dataverse
14:31 poikilotherm
Ah now I remember - JaCoCo tries to add itself to the argLine, but it was thrown out because of the argLine config for surefire
14:32 poikilotherm
Before this line did not exist
14:32 poikilotherm
And I needed to add the variable so the Coverage works
14:32 poikilotherm
Ok let me check how to get around this
14:33 poikilotherm
Maybe I can set some properties so JaCoCo is happy and Netbeans does not need to escape stuff :-/
14:34 pdurbin
poikilotherm: ok, do you mind if I assign the issue to you?
14:35 poikilotherm
Carry on :-D
14:37 pdurbin
thanks!
14:44 cdsp-rmo joined #dataverse
14:45 poikilotherm
Ok, I think I got it. Moved it from plugin config to properties.
14:46 poikilotherm
Unit tests work properly, will need to run IT tests later. Need to go in a minute, maybe adding this later this evening (read: afternoon at EDT...)
14:46 cdsp-rmo
hello world
14:46 pdurbin
poikilotherm: thanks!
14:47 pdurbin
cdsp-rmo: how's it going?
14:47 cdsp-rmo
fine ! And you ?
14:48 pdurbin
not bad. the frame of my bike broke but in theory there's a lifetime warranty. expecting a call today
14:48 cdsp-rmo
:o
14:48 donsizemore
pothole?
14:48 pdurbin
There are no potholes in Boston.
14:48 pdurbin
Perfect weather too.
14:49 pdurbin
donsizemore: my wife and kids loved that video you linked to, by the way
14:49 cdsp-rmo
I have a little question about the json import api in the native api
14:49 cdsp-rmo
why is it superuse restricted ?
14:49 cdsp-rmo
(I mean, the new api import function)
14:50 pameyer
cdsp-rmo: because it's designed for migrating a repository from one software stack to dataverse
14:50 cdsp-rmo
okay
14:51 cdsp-rmo
I was thinking about combining this api point with the xml one (so that you could put a ddi xml or a json as a file input)
14:51 cdsp-rmo
but the superuser restriction may not be optimal, so I think it will be a "nope" ^^"
14:58 pdurbin
cdsp-rmo: in a lot of cases, we'll start with a superuser only API but eventually that API is opened to non-superusers. And eventually, maybe a GUI would be built. For example in Dataverse 4.9.3 it's now possible for non-superusers to link datasets: https://github.com/IQSS/dataverse/issues/4410
15:08 pdurbin
Does that make sense?
15:10 djbrooke joined #dataverse
15:14 djbrooke joined #dataverse
15:15 djbrooke_ joined #dataverse
16:17 djbrooke joined #dataverse
16:26 djbrooke joined #dataverse
16:57 dataverse-user joined #dataverse
17:00 donsizemore joined #dataverse
17:21 djbrooke joined #dataverse
17:25 dataverse-user
@pdurbin. I'll have a detailed look at the restricted files. Thanks a lot for your help.
17:54 donsizemore
@pdurbin i got the commit syntax working, but have another dumb question
17:55 donsizemore
@pdurbin n/m, i found it. see, i just had to pester you!
18:12 djbrooke joined #dataverse
18:21 pdurbin
:)
18:34 djbrooke joined #dataverse
18:35 pameyer
pdurbin: thanks, good catch
18:36 pdurbin
sure
18:46 pdurbin
pameyer: d511.zip 95%[++++++++++++++++++> ] 244.88G 16.3MB/s eta 15m 16s
18:47 pameyer
pdurbin: that one uploaded faster than I'd expected it to
18:47 pameyer
do you have any free space left on your hd? ;)
18:47 pdurbin
not much :)
19:11 donsizemore joined #dataverse
19:17 jri joined #dataverse
19:32 Julio joined #dataverse
19:37 Julio
Hi! I'm developing a project to use dataverse in my institution. I'm looking for a training on the platform. Do you have a guide on how could I found some training?
19:38 pdurbin
Julio: hi! You should email support
19:39 Julio
Nice! Thank you !
19:41 pdurbin
sure
19:46 pdurbin
pameyer: good times with sed
19:48 jri joined #dataverse
19:50 pameyer
well, I tried tr first and couldn't get the incantation right
19:50 pdurbin
it's tricky
19:51 pdurbin
pameyer: I'm calculating the md5 for d511.zip
19:51 pdurbin
fans spinning up. 50% cpu used by md5
19:52 pameyer
flashbacks to doing parallel checksumming in java - one of the first times I learned that garbage collection doesn't mean that you don't have to close your file descriptors
19:53 pdurbin
donsizemore: I just ran `ec2-create-instance.sh -r https://github.com/TexasDigitalLibrary/dataverse -b IQSS-5095-Inherit_admins` but Dataverse wasn't deployed to http://ec2-54-86-53-41.compute-1.amazonaws.com:8080/ ... just the Glassfish page is shown.
19:54 donsizemore
@pdurbin depending on when the branch was created, the hard-coded version in pom.xml might not be current. i've been shaking my fist at ansible's XML module most of the afternoon
19:54 pdurbin
donsizemore: the branch was created before we bumped the version in pom.xml from 4.9.2 to 4.9.3
19:55 donsizemore
@pdurbin yup. i'm opting to fight with the ansible's xml module instead of making pom.xml a template because of the frequency of updates to pom.xml
19:55 pdurbin
pameyer: my fans stopped. I looked over at my terminal with `md5 d511.zip` running in it and it exited with "Killed: 9". :(
19:56 donsizemore
@pdurbin (unless i'm misunderstanding)?
19:57 pdurbin
donsizemore: buh. I want to help but I'm confused about what the problem is.
19:57 pdurbin
Is there a line of code you can point me to?
19:57 donsizemore
so, i believe(?) the warfile takes its name from the version in pom.xml
19:58 donsizemore
dataverse-ansible has a configurable "version" group_var, which is now 4.9.3, so it looks for dataverse-4.9.3.war but maven builds 4.9.2 from the warfile
19:58 donsizemore
and ansible's XML module is surprisingly... give me a minute and i'll pick an adjective
19:58 pdurbin
:)
19:59 pdurbin
yes, the warfile gets its name from pom.xml
19:59 donsizemore
the simple thing to do would be to make a jinja template of pom.xml, but it's updated pretty frequently
20:01 donsizemore
@pdurbin i'd love a second pair of eyes, as all i can seem to do is make xpath angry. i'm giving it:
20:01 donsizemore
- name: depending on branch, the project version in pom.xml doesn't always match. xml: path: "{{ dataverse.srcdir }}/pom.xml" namespaces: ns: http://maven.apache.org/POM/4.0.0 xpath: ns:project/version attribute: version value: "{{ dataverse.version }}" pretty_print: yes
20:03 pameyer
xpath is always angry
20:05 pdurbin
donsizemore: does this help? VERSION_FROM_POM=$(cat pom.xml | xmlstarlet sel -N p='http://maven.apache.org/POM/4.0.0' -t -v '/p:project/p:version')
20:05 donsizemore
@pdurbin i've tried so many permutations... and i just need to change one numeral in the file (today, anyway)
20:06 pdurbin
Ok, well that xmlstarlet command above is what I use in one of our Jenkins jobs.
20:09 donsizemore
it looks good, and makes sense. i'm getting payback for calling JSON "XML for Millenials"
20:10 pdurbin
That reminds me, I just heard that JSLT is a thing: https://github.com/schibsted/jslt
20:16 donsizemore
i think i'm going to do something really ugly with sed instead
20:17 pdurbin
donsizemore: I'm still confused what the root problem is but let me re-read what you wrote above.
20:19 pdurbin
donsizemore: what if we do a better job of setting expectations of which branches can be spun up? Only those who have the same version in pom.xml as the latest release? Or whatever. Would that help? I asked Jim to merge the latest from develop into his branch already.
20:50 pameyer
if I'm remembering right; xmlstar has different syntax for find than find/replace
20:50 pameyer
but I've also wondered *why* the war file needs a version number in it
21:38 donsizemore joined #dataverse
21:39 donsizemore
@pdurbin @pameyer i knew i should've gone to the gym earlier. i'm simply going to drop the 'develop' version of pom.xml on top of whatever branch we have. it would clear up the version screwiness, and can you think of any instance in which we'd be better off compiling from an out-of-date pom.xml vs. current?
21:41 pameyer
@donsizemore - I can see that possibly breaking things between branches on dependency updates, but that might be a tradeoff worth making
21:42 donsizemore
@pameyer i would think a developer would want to know what in a given branch would break when merged into develop, and dependency versions would be a great thing to test
21:44 pdurbin
donsizemore: out of date pom.xml? I'm confused. When running `mvn package` we always want to use the pom.xml from the branch, right?
21:44 donsizemore
@pameyer and after an afternoon of reading about ansible, xml and xpath i'm in forgiveness/permission mode
21:44 pameyer
@donsizemore - I agree, it sounds like the best approach
21:44 pameyer
it'll break stuff; but I think that's stuff that is reasonable to break - especially given xpath
21:45 pameyer
my opinion of dependency versions is actually worse than my opinion of xpath
21:45 donsizemore
@pdurbin it's really just the version string in pom.xml that's breaking things, but for now i'm leaning toward pulling the pom.xml from develop and seeing what detonates
21:46 pdurbin
seems weird to me to always pull it from develop when you have the branch in your hands but whatever works :)
21:47 donsizemore
@pdurbin the XML module is under heavy development. maybe i'll use it in 2.7
21:47 pameyer
building on the deploy box seems weird to me ;)
21:49 pdurbin
pameyer: yeah, and it's slow to build on the deploy box. but we have to build it somewhere :/
21:51 pameyer
~800 MB of maven cache .....
21:51 * pdurbin
cries
21:51 pdurbin
T-T
21:52 donsizemore
@pdurbin if you want i can read in the value from 'nproc' and launch 2 threads per core in the playbook?
21:52 pdurbin
would that help? sure!
21:52 pdurbin
I mean, we're paying for a "medium", right? :)
21:53 donsizemore
actually i think i can just give it -T 1C
22:01 pameyer
I was over estimating - build container only needs ~200MB of maven cache
22:02 donsizemore
the bulk of the time is spent downloading All the Things
22:09 pameyer
pdurbin: 4698076a52c7c8de3f4b1f39cbc022b7 should be the md5 you're looking for
22:11 pameyer left #dataverse
22:15 donsizemore
@pdurbin i think i have all this working, but i want to test some more this evening (and i want to go home)
22:15 donsizemore
@pdurbin so i promise a commit tonight or first thing in the morning?
22:16 donsizemore
@pdurbin n/m, pushed what i think is working, will test this evening =)
