Time
S
Nick
Message
04:46 iamtimmo joined #dataverse
07:09 jri joined #dataverse
08:23 bricas joined #dataverse
08:48 jeffspies______ joined #dataverse
11:06 Haridass joined #dataverse
11:06 Haridass
Good Morning @pdurbin
11:07 Haridass
Finally Dataverse is up and running in CentOS 7
11:09 pdurbin
Haridass: nice! Good job.
12:05 donsizemore joined #dataverse
13:15 bsilverstein joined #dataverse
13:32 djbrooke joined #dataverse
13:33 djbrooke joined #dataverse
13:46 Aruna joined #dataverse
13:46 Aruna
Hi @pdurbin, I am Aruna from Haridass team
13:46 pdurbin
Aruna: hi!
13:49 Aruna
I registered as a user in our local dataverse installation.
13:50 Aruna
and dataverseAdmin (superuser) has granted me contributor role
13:50 Aruna
Still, I am not seeing the buttons to add data
13:51 Aruna
in the page http://192.168.3.250:8080/dataverse.xhtml
13:51 pdurbin
Let's look at http://guides.dataverse.org/en/4.3.1/user/dataverse-management.html#permissions
13:52 pdurbin
Aruna: which radio button was selected under "who can add to this dataverse?" http://guides.dataverse.org/en/4.3.1/user/dataverse-management.html#permissions
13:52 pdurbin
whoops. here are the radio button choices: http://guides.dataverse.org/en/4.3.1/_images/dv3.png
13:54 Aruna
first radio button i.e., Anyone adding to this dataverse needs to be given access
13:54 Aruna
is selected
13:55 pdurbin
Aruna: please have your superuser select one of the other options so that you can see the "Add Data" button.
13:55 pdurbin
Aruna: you can show your superuser this page: http://guides.dataverse.org/en/4.3.1/installation/config.html#root-dataverse-permissions
14:01 pdurbin
djbrooke: another account lockup scenario: https://github.com/IQSS/dataverse/issues/3153#issuecomment-226773488
14:01 djbrooke
ahhahaha
14:05 pameyer joined #dataverse
14:09 djbrooke_ joined #dataverse
14:10 pdurbin
donsizemore: morning. Do you have a shibd config file with a feed *and* and a static XML file?
14:11 donsizemore
@pdurbin i'm happy to send you the config from dataverse-test, which temporarily had both (and everything is still present, just commented out)
14:12 pdurbin
donsizemore: yes, if you would email it to me I'd really appreciate it.
14:12 pdurbin
donsizemore: do you plan to do any side deals with Emory or Harvard?
14:12 donsizemore
@pdurbin: more importantly, jon just asked me to come to the community meeting (akio can't make it)
14:13 pdurbin
nice! it'll be a good time
14:13 donsizemore
@pdurbin: i'm waiting to hear back from emory and welcome a side deal with harvard
14:13 pdurbin
donsizemore: ok, so you very well may roll with the feed+file config in production if a side deal comes through.
14:13 donsizemore
@pdurbin: the first suggested hotel on you all's' website is the holiday inn, but not the not the inn at longwood
14:14 donsizemore
@pdurbin: i value proximity to good asian/indian food and pipe organ recitals... any recommendation on where to book?
14:15 donsizemore
@pdurbin: the only issue i ran into running off local/dynamic shib IdP lists were from "duplicate" accounts when we changed up the account and existing users tried to sign back in
14:15 pdurbin
buh. please email support
14:15 pdurbin
:)
14:16 donsizemore
@pdurbin: it would be a matter of converting them to local, re-assigning ownership of any uploaded files, and whatever else.
14:16 pdurbin
donsizemore: hmm, so you know how to fix it? You don't need a code change?
14:16 donsizemore
er, ^^ when we changed the shib config
14:19 pdurbin
donsizemore: I'm not quite following. Mostly I want to know if you feel like it's a bug or not.
14:20 pdurbin
pameyer: closer, closer to what we want. :)
14:20 donsizemore
@pdurbin: it's an ID mgmt thing, not a bug in Dataverse. Shib identifies people by e-mail address and since say jonathan_crabtreeDB
14:21 donsizemore
@pdurbin: any Dataverse admins would only run into that situation during testing, as I don't imagine they'd swap IdPs in and out regularly
14:21 pdurbin
hmm, but shouldn't Dataverse prompt him to convert his account?
14:21 pdurbin
rather than crabbing about Crabtree :)
14:22 pdurbin
(hi, Jon!) :)
14:22 donsizemore
@pdurbin: i wondered why it didn't just keep him, since he was signing in from the same SSO
14:22 donsizemore
@pdurbin but Shib and/or Dataverse saw him as a different person, probably due to the EntityID in Shib(?)
14:23 pdurbin
You're right that Shib identifies users for *conversion* from local to shib based on email address. Baed on the email address asserted by the IdP.
14:27 donsizemore
@pdurbin sent
14:28 pdurbin
donsizemore: starred in gmail. thanks!
14:29 donsizemore
@pdurbin: aha: the e-mail was the same but the affiliation was different
14:29 donsizemore
our little federation called us "UNC-Chapel Hill" but InCommon calls us "University of North Carolina at Chapel Hill"
14:30 donsizemore
in other words, I think it was a "me" problem
14:30 pdurbin
well, the logic for prompting for account conversion from local to shib only pays attention to email address, not affiliation
14:36 donsizemore
@pdurbin: also, i've found mei mei, fugakyu and sol azteca within walking distance of the holiday inn, so i think i've found my spot =)
14:37 pdurbin
oh, that holiday inn is only a couple blocks from my condo
14:37 pdurbin
nice neighborhood :)
14:40 pdurbin
pameyer: body("data.script", startsWith("#!") <-- assert that script from DCM starts with a shebang :)
14:44 djbrooke joined #dataverse
14:45 djbrooke joined #dataverse
14:47 pdurbin
donsizemore: you're famous by the way: https://youtu.be/r7m9nDAcYu0
14:48 djbrooke_ joined #dataverse
15:05 pameyer
pdurbin: nice!
15:06 pameyer
also asserting that it ends with EOF?
15:06 djbrooke joined #dataverse
15:06 pdurbin
meh!
15:18 pdurbin
pameyer: I've been thinking about ur.py and sr.py... should I be specifying to the Data Capture Module that I want *rsync* rather than something else, such as Globus?
15:18 pameyer
good question
15:19 pameyer
I'd been thinking that there would be different dcm's for different protocols
15:19 pdurbin
orly
15:19 pameyer
yup - but that might be something better abstracted away
15:20 pdurbin
here's where a diagram might help :)
15:20 pameyer
I could add some boxes to the whiteboard drawing ;)
15:20 pameyer
but yeah, a diagram might be helpful
15:21 pdurbin
or a glossary
15:21 pameyer
we've already got a list of abbreviations - should I update it? :)
15:22 pdurbin
Oh! What is that doc linked from?
15:22 pameyer
do you think it makes sense for dataverse to keep track of things like "ask dcm1 for rsync+ssh, dcm2 for globus"?
15:22 pameyer
I'm thinking that it doesn't
15:22 pameyer
but still somewhat hazy
15:23 pameyer
I think the requirements doc started with a list of abbreviations
15:23 pameyer
doesn't have dcm in it yet; seem like it wasn't being looked at
15:23 pdurbin
let's keep the description of https://github.com/IQSS/dataverse/issues/3145 up to date with useful links to glossaries and such
15:26 pdurbin
pameyer: what does the DCM respond with if I ask for an rsync script for a "normal" dataset? An error? "Sorry, you can't have an rsync script because this is a 'normal' dataset."
15:27 pameyer
it assumes dataverse knows what it's doing, and gives it an rsync script
15:28 djbrooke joined #dataverse
15:30 bsilverstein
pdurbin: any way I could hop onto that team calendar on google?
15:34 pdurbin
djbrooke: do you have the power to give bsilverstein access? I'm not sure I do.
15:36 pdurbin
pameyer: I'm fine with that for now but I'm leaving a todo in the code to revisit this.
15:37 pameyer
pdurbin: or we add protocol=rsync+ssh to the ur.py request, and update the docs in that repo
15:37 pameyer
either way works for me
15:38 pdurbin
pameyer: I think I'd like the Data Capture Module to query Dataverse if necessary to determine if an rsync script should be created for a particular dataset or not.
15:39 pameyer
why?
15:39 djbrooke
bsilverstein: done
15:39 pdurbin
djbrooke: thanks!
15:39 pameyer
not opposed, just not understanding
15:40 pdurbin
pameyer: let me think about it some more. Again, I'm leaving todo's in the code. :)
15:40 pameyer
works for me
15:41 bsilverstein
djbrooke: sweet, thanks
15:58 djbrooke_ joined #dataverse
15:59 djbrook__ joined #dataverse
16:13 donsizemore joined #dataverse
16:15 iamtimmo joined #dataverse
17:36 djbrooke joined #dataverse
17:38 bsilverstein joined #dataverse
18:06 djbrooke joined #dataverse
18:28 djbrooke joined #dataverse
18:38 djbrooke joined #dataverse
18:45 pameyer joined #dataverse
18:45 djbrooke joined #dataverse
19:13 donsizemore joined #dataverse
19:13 donsizemore
@pdurbin knock knock?
19:23 pdurbin
donsizemore: who's there
19:24 donsizemore
@pdurbin: what do you know about Dataverse's DSB timer and failed jobs restarting automatically? we'd like to clear them
19:25 pdurbin
DSB?
19:26 donsizemore
thumai is trying to ingest an 851MB file. it fails with Caused by: java.lang.OutOfMemoryError: Requested array size exceeds VM limit
19:26 pdurbin
ew
19:27 donsizemore
i stopped glassfish to bump the heap size to 132G (from 48G) and on start-domain all four failed jobs appeared to restart automatically. we don't want them to keep doing this over the weekend
19:27 donsizemore
akio and i are looking through the console options, but wherever the failed jobs are stored... i want to nuke them
19:28 donsizemore
(even with the Xmx value at 132G, btw, the job failed with an out of memory error, even though the machine's memory usage never climbed that high)
19:29 pdurbin
Data Services Broker
19:30 pdurbin
donsizemore: dunno, sorry. Please email support
19:33 donsizemore
as long as it doesn't keep relaunching them over the weekend...
19:34 balo joined #dataverse
19:47 pdurbin
pameyer: if the Data Capture Module sends "validation failed" do you want Dataverse to respond to the DCM with 200 and "ok, I got your message"? (And I'll go notify the user.) Or do you want a non-200 response?
19:47 djbrooke joined #dataverse
19:48 pameyer
pdurbin: this is sounding like a familiar pattern. how often can/should dcm tell dv that there's a problem?
19:49 pdurbin
well, I'm stubbing out code to listen to messages
19:53 pdurbin
pameyer: you can send whatever messages you like :)
19:53 pdurbin
pameyer: I'll return the code for internal server error if I'm unable to notify the user on "validation failed"
19:57 djbrooke joined #dataverse
20:00 pameyer
that works for me
20:02 pdurbin
cool
20:02 pdurbin
pameyer: I think we're going to need a new notification type. The current list: ASSIGNROLE, REVOKEROLE, CREATEDV, CREATEDS, CREATEACC, MAPLAYERUPDATED, SUBMITTEDDS, RETURNEDDS, PUBLISHEDDS, REQUESTFILEACCESS, GRANTFILEACCESS, REJECTFILEACCESS
20:02 pameyer
should I keep trying until I get 200?
20:03 pdurbin
hmm, I guess. please ask me again later :)
20:03 pameyer
I think you're right on type - I don't see anything on that list
20:04 pdurbin
CHECKSUMFAIL?
20:05 pameyer
seems reasonable
20:05 pdurbin
cool
20:06 pdurbin
pameyer: I may want to flip flop on something I said the other day. For now it would be easier to send the notification only to the user who initially made the upload request.
20:18 pameyer
pdurbin: I don't have a strong reason to go one way or the other yet
20:19 pdurbin
ok, cool. I'm leaving a todo in the code to revisit this.
20:19 pameyer
I'm wondering if there's anything we can borrow from "ingest failed" or existing "upload failed" ...
20:20 pameyer
more something that just occured to me than anything I've got a good example of something to borrow
20:20 pdurbin
hmm, maybe
20:42 pdurbin
pameyer: ok, I gotta run but I've got code to show Bill next week at https://github.com/pdurbin/dataverse/tree/3145-dcm ... especially about how "validation passed" should kick off his file system crawler to import files
20:43 pdurbin
have a good weekend all. I'm home with the kids on Monday
20:43 pdurbin left #dataverse
20:53 djbrooke joined #dataverse
21:38 djbrooke joined #dataverse
23:51 djbrooke joined #dataverse
