Time
S
Nick
Message
00:48 jri joined #dataverse
04:49 jri joined #dataverse
06:51 juancorr joined #dataverse
07:06 jri joined #dataverse
07:15 jri_ joined #dataverse
09:06 cdsp-rmo joined #dataverse
10:05 jri joined #dataverse
10:09 jri_ joined #dataverse
10:13 pdurbin joined #dataverse
10:17 jri joined #dataverse
10:27 jri joined #dataverse
11:37 cdsp-rmo joined #dataverse
12:03 donsizemore joined #dataverse
12:55 cdsp-rmo
pdurbin: I think I know where the problem is for the multiple values
12:55 cdsp-rmo
schema.xml file for solr has to be edited too, no ?
12:56 cdsp-rmo
(with the merge, I can pop new fields on the ui for depositor, but the dataset cannot be created, provoking an error on solr side)
12:57 cdsp-rmo
(and hello all, by the way :D)
12:57 pdurbin
cdsp-rmo: ah, yes, you're right. Multiple value affects Solr.
12:59 cdsp-rmo
ok
12:59 cdsp-rmo
so, I changed solr and postgres sides
13:00 cdsp-rmo
I can have 2 depositors for a dataset via the gui
13:00 cdsp-rmo
yeaaaah
13:00 cdsp-rmo
but my ddi import doesn't work, got an error: "Error parsing datas as Json: incorrect multiple for field depositor"
13:00 cdsp-rmo
has to figure out why :o
13:01 pdurbin
Hmm. I'm not sure.
13:02 cdsp-rmo
parseDataset from jsonParser I guess
13:02 cdsp-rmo
checking
13:19 donsizemore
@pdurdin morning... and knock knock?
13:20 pdurbin
donsizemore: mornin. What can I do for you?
13:21 Jim__ joined #dataverse
13:36 juancorr joined #dataverse
13:42 donsizemore joined #dataverse
13:43 donsizemore
@pdurbin good morning to you, sir! i'm pestering because i believe we're hitting #4565. however, we _do_ have a test dataverse holding production data which i've upgraded to 4.9.4
13:44 donsizemore
@pdurbin so, my question: if i ask the user to reset her password there, then copy the hash back into our 4.8.6 database... decently safe / solid?
13:45 pdurbin
should work
13:45 pdurbin
could try it with a test account first
13:54 donsizemore
@pdurbin or is there a more straightforward way for me to reset the user's password so she can change it? i do hate the temporary password thing, though, because they never change them.
13:55 pdurbin
What happens when the user tries to reset her password in production? Any details about how it fails?
13:56 pdurbin
Any does password reset work in general?
13:58 donsizemore
@pdurbin i don't see dataverse (or the system maillog) sending out the reset e-mail, which led me to #4565
13:58 pdurbin
You can bump up the logging and see the password reset URL in server.log
14:00 pdurbin
I'm not sure if you've messed with logging levels before but an example can be found at http://guides.dataverse.org/en/4.8.6/developers/debugging.html
14:08 cdsp-rmo
pdurbin: I get it why it doesn't work. multiplicity of fields are hardcoded in the ddi import process
14:09 cdsp-rmo
so, database fieldtype is good, but the fieldtype emulated by the DTO dataset (created by the ImportDDIServiceBean) is not, so that's why there is an error
14:09 cdsp-rmo
:S
14:11 pdurbin
Hmm. Can you make the DDI import support both a single depositor and multiple depositors?
14:18 cdsp-rmo
well
14:18 cdsp-rmo
I guess the best way would be to check the fieldtype allowmultiplevalue in the database
14:18 cdsp-rmo
and not just put "false" or "true" in the code "because"
14:20 pdurbin
Sure, that makes sense. I guess there's an impedance mismatch between the ability to change "has multiple" in the database and hard coded assumptions about if a field is multiple or not.
14:20 juancorr joined #dataverse
14:21 pdurbin
I wonder if other parts of that DDI import code pay attention to what the database says. Maybe there are a lot of hard coded assumptions. I don't know.
14:22 cdsp-rmo
the DDI import use the FieldDTO methods (which can be multiple, single, vocab, etc)
14:22 cdsp-rmo
and the DDIImport decide which one to use, and doesn't care about what the database has
14:23 cdsp-rmo
it's only when you want to create the dataset with the DTODataset generated to json that you got the problem of different FieldTypes
14:24 cdsp-rmo
I guess the best way to handle this is to have a method that check in the database, for a typename given, what are the attributes for the field
14:24 cdsp-rmo
and then, which method of datasetDTO to use (multiple, single, voca, etc)
14:24 donsizemore
@pdurbin that was an excellent suggestion, but unfortunately i can see the URL only in our 4.9.4-versioned dataverse-test
14:25 donsizemore
@pdurbin our production 4.8.6 dataverse crabs that her e-mail doesn't exist because she's not in builtinuser
14:25 pdurbin
cdsp-rmo: that makes sense but before writing too much code, it would probably be best for you to explain the problem in the issue and/or write to https://groups.google.com/forum/#!forum/dataverse-dev
14:25 cdsp-rmo
ok
14:26 cdsp-rmo
should I update the allowmultiple github issue I made ?
14:26 cdsp-rmo
about the solr file to edit
14:26 pdurbin
donsizemore: it's normal for users to not be in the builtinuser table if they are shib or oauth users. Is she one of those? Was there a failed conversion?
14:27 pdurbin
cdsp-rmo: yeah, I think you should leave a comment at https://github.com/IQSS/dataverse/issues/5164 with what you've learned
14:28 pameyer joined #dataverse
14:29 pdurbin
cdsp-rmo: what's the status of https://github.com/IQSS/dataverse/pull/5170 ? Do you want us to QA and possibly merge it even if it doesn't support multiple depositors? Or do you want us to wait?
14:30 cdsp-rmo
as a first part of the ddi import, I guess it's still good
14:30 cdsp-rmo
changing multiplicity can come later
14:30 cdsp-rmo
imho
14:31 cdsp-rmo
it just means that "every" xml coming from nesstar won't be 100% ok, but a good portion will be ok (and there will be clear messages about the field causing problems, to edit the files if needed)
14:34 pdurbin
Ok, I just moved it to QA. Thanks.
14:34 cdsp-rmo
(if it's ok, I can update the pull with latest develop and push it)
14:34 pameyer
@donsizemore I've had mixed results trying to copy hashes around in the db as a way to handle different passwords. likely something I was missing, but I wasn't able to get it to work consistently
14:35 pdurbin
cdsp-rmo: sure, please go ahead
14:35 cdsp-rmo
@pameyer take care of special chars auto escape (especially double ones), sometimes it's very vicious ...
14:36 pdurbin
donsizemore: oh, you're saying she's in the builtinuser table but her email is missing. What if you add it?
14:52 cdsp-rmo
pdurbin: oops, I closed and coment https://github.com/IQSS/dataverse/issues/5164 instead of just comment. Should I reopen the issue or we consider it closed ? (given that it's working now, except for my ddi import but ... it's special))
14:54 pdurbin
Well, you should decide if you want multiple depositors to be supported or not.
14:54 Jim__ joined #dataverse
14:55 cdsp-rmo
maybe an other issue ?
14:55 cdsp-rmo
or an other pull request
14:55 cdsp-rmo
ah
14:55 donsizemore joined #dataverse
14:56 cdsp-rmo
I get it. The issue was to see how to do that in my head, but not at all
14:56 cdsp-rmo
gonna reopen it :o
14:56 pdurbin
that's fine
14:57 donsizemore
@pdurbin that's the weird thing. i don't think she's a shib user as her affiliation isn't populated and i don't find her university (united nations in hong kong) in our drop-down
14:58 donsizemore
@pdurbin she's in authenticateduser but not builtinuser, and when i try to manually trigger a password reset for the e-mail address from the database, dataverse 4.8.6 crabs "Couldn't find single account using kouba
15:00 pdurbin
donsizemore: sorry, I think I'm getting turned about. Is she in the builtinuser table at all?
15:02 donsizemore
@pdurbin she is not.
15:04 donsizemore
@pdurbin i may open a support ticket about this, just to track down the information that i do have about her, and be recommended the safest way to update her password manually?
15:04 donsizemore
@pdurbin it _would_ work if we were on 4.9.4, but we're not yet pending archive staff testing (and availability of new digs in our campus research computing infrastructure)
15:07 pdurbin
donsizemore: a support ticket would probably be best. I feel like we've had to deal with this before.
15:07 pdurbin
royal we
15:08 * donsizemore
bows, majesty.
15:11 pdurbin
heh
15:11 pdurbin
donsizemore: have you tried dumping out her user record as JSON ?
15:20 * donsizemore
searches API docs
15:22 donsizemore
@pdurbin she exists "authenticationProvider":"BuiltinAuthenticationProvider"
15:24 pdurbin
interesting
15:24 pdurbin
but no entry in the builtinuser table
15:24 pdurbin
failed coversion?
15:25 pdurbin
something must have gone wrong
15:25 pdurbin
Does password reset work fine for most users?
15:27 donsizemore
@pdurbin shrug. i'm shibbolized
15:29 Jim__
I'm seeing kcondon's note on the community list - is it really known that you have to deploy intermediate wars or is that still a guess? When I was looking, I just couldn't see a mechanism for that to be true...
15:29 donsizemore
@Jim__ I wouldn't dare upgrade without stepping through the point releases
15:31 Jim__
That's clearly a good practice, but I'm wondering if the war part is too conservative or if there is a known issue. (Clearly the db update scripts, etc. have to be applied in order, etc.)
15:31 pdurbin
Jim__: check out https://github.com/IQSS/dataverse/issues/5204 ... a head scratcher for me
15:31 Jim__
As far as I can tell, if an intermediate should create a table, the next version will do it as well if that's the first war you deploy...
15:32 Jim__
Right - I've seen things that hint at it, but I haven't yet heard of one where the root cause is known.
15:33 pdurbin
Jim__: so are you with me or Slava on that issue? :)
15:36 pameyer
intermediate deploy $x creates table, deploy $x+$i tries to alter table...
15:37 Jim__
pdurbin - I think I see all those failures to recreate existing tables, so I think I agree with you that they should be there...
15:37 pameyer
not sure if that's an actual problem, but that's something I could see as being a failure mode when not stepping through things
15:38 Jim__
pameyer - how does the next version know its not the first deployed? I think it just tries to create the table again (with a different structure if it has changed between versions - which is why we need update scripts - the create of an altered table fails )
15:40 Jim__
I actually saw the opposite - I did a 4.8.4 to 4.8.6 update and saw the 4.8.5-4.8.6 update script fail because the new table (which would have been created at 4.8.5) didn't exist for the update to work on it, but the 4.8.6 war deploy created the table in its final form
15:40 pameyer
Jim__ not sure. haven't dug into it too much yet
15:45 pameyer
I tend to end up sticking on the "try something, but don't worry if it fails" bits - hasn't yet been high urgency for me though
15:59 Jim__
pdurbin - commented on 5204 - I can delete the table and the 4.9.4 install recreates it...
16:08 cdsp-rmo joined #dataverse
16:51 pdurbin
Jim__: thanks. So you did deploy the war file in between. I don't know. It's an odd bug report to me.
16:52 Jim__
for 4.8.4-4.8.6 I did not. For the one I commented on, we did deploy 4.9.3, but I manually went in and deleted the table (still empty for us) and saw it get recreated by the 4.9.4 war deploy.
16:54 pdurbin
Right, I think we're on the same page. I think we have the same mental model of when it's necessary to deploy the war file.
16:54 pdurbin
Jim__: unrelated, I just left a comment here after Kevin asked me to take a look: https://github.com/IQSS/dataverse/issues/5044#issuecomment-431895375
17:03 donsizemore joined #dataverse
17:37 donsizemore joined #dataverse
18:34 Jim__
Another odd topic - anyone seeing crawls from pipl, vultr, and/or semrush? I'm guessing they are after schema.org metadata. I hadn't heard of them but see them on QDR and TDL...
18:38 pdurbin
Good question for people who run Dataverse in production and might be paying attention to logs. cdsp-rmo donsizemore juancorr ^^
18:38 pdurbin
andrewSC bricas: I forget if you're in production yet or not
18:39 pdurbin
pameyer: soon! :)
18:39 pameyer
definately seen semrush
18:40 pameyer
hmm - and BLEXBot looks like it ignores robots.txt....
18:40 donsizemore
@Jim__ our most frequent ipv4 hit is the campus security scanner ;-)
18:40 donsizemore
@Jim__ followed closely by GoogleBot
18:42 andrewSC
pdurbin: you could consider it prod lol
18:42 andrewSC
prod to us is 24/7 availability + org wide use but not necessarily publicly exposed, yet
18:43 pdurbin
andrewSC: ok, when you're ready, please see http://guides.dataverse.org/en/4.9.4/installation/config.html#putting-your-dataverse-installation-on-the-map-at-dataverse-org :)
18:43 donsizemore
@Jim__ I do see the semrush crawler pulling our robots.txt file (which I wiped out)
18:43 andrewSC
duly noted!!
18:44 donsizemore
@andrewSC when are we going to have a Dataverse mini-conference?
18:44 andrewSC
i suspect we'll be on the map early next year? Whenever the DoD mandates we expose some research (iirc)
18:44 andrewSC
lol
18:44 andrewSC
idk!
18:45 donsizemore
@andrewSC I would like it noted that Town Hall Burgers, in neutral Dukie country, serves blood orange bold rock cider on tap
18:46 andrewSC
interesting
18:46 andrewSC
out past cary
18:46 andrewSC
not too far from me though!
18:46 andrewSC
I'll have to go check it out sometime--looks like good eats!
18:48 pameyer
hmm... BLEXBot claims it respects robots.txt, but the evindence shows that is a lie.
18:55 Jim__
thanks!
19:09 pameyer
hope blexbot enjoys it's 403 responses
19:15 donsizemore
so @pdurbin... ze plot thickens, lahk ze bad rue
19:16 pdurbin
rue? street?
19:17 donsizemore
^roux
19:18 pdurbin
yum
19:19 donsizemore
sabrina says she's never used institutional login, or converted, but i have no builtinuser.username corresponding to her authenticateduser account. she has a gmail account but has never attempted to convert it and wants to keep it separate. so... i'm wondering where her builtinuser entry went
19:20 donsizemore
(if i were a BOFH i'd go ahead and upgrade dataverse, which would drop the builtinuser attribute columns in the process)
19:20 pdurbin
heh
19:20 pdurbin
she only has one entry in authenticateduser?
19:21 donsizemore
in the mean time, she nor i can easily set her password because she's not there.
19:21 donsizemore
i did notice the "can't find single entry" verbage
19:21 donsizemore
but her account was created after our 3.6 => 4.5 migration, so no duplicate username
19:22 pdurbin
does the user dashboard help at all?
19:23 donsizemore
she only shows up once, using the e-mail account in question
19:23 pdurbin
at the one instance is a builtin user? but has no entry in the builtinuser table? that's not right
19:24 donsizemore
correct, she's in authenticated but not builtin
19:24 pdurbin
upgrading isn't going to help. the builtinuser table is still around, we're just dropping some duplicate columns from it
19:25 donsizemore
i'm-a-skeert to create an entry for her myself... fiddling with user info was how i rendered my first dataverse 4 instances inoperable during shibboleth testing
19:25 pdurbin
heh
19:26 pdurbin
does she have any data? any roles?
19:26 donsizemore
no data, file downloader role
19:26 pdurbin
for a lot of files?
19:27 donsizemore
dunno but she's the only entry in authenticateduser from unu.edu
19:28 pdurbin
does her assigneeidentifier show up much in the roleassignment table? http://phoenix.dataverse.org/schemaspy/latest/tables/roleassignment.html
19:29 donsizemore
zero i think
19:30 pdurbin
If there's no trace of her in the system, you might consider deleting her account with an API call. Then have her create a fresh account.
19:31 donsizemore
that's what she and mandy each asked, but i'm not yet certain there's no trace of her =)
19:37 pdurbin
donsizemore: there's "List all role assignments of a role assignee (i.e. a user or a group)" at http://guides.dataverse.org/en/4.8.6/api/native-api.html
19:45 donsizemore
that endpoint only seems to return OK, "data" even for our admins? (most likely, i haz a stupid)
19:46 donsizemore
n/m, got it to return for thu-mai
19:50 pdurbin
phew
19:51 donsizemore
you'd think i'd know her username != her UNC account ID by now
19:52 pdurbin
:)
19:52 pdurbin
any role assignments for her?
20:46 pameyer joined #dataverse
21:16 donsizemore
@pdurbin none for sabrina that i can see
21:23 pdurbin
donsizemore: oh, I forgot. There should be some sort of "remove role assignments" button in the user dashboard.
21:23 donsizemore
@pdurbin i saw that =) i wouldn't be so squeamish if jon hadn't just flown to botswana for RDA
21:23 pdurbin
:)
21:40 jri joined #dataverse
22:27 pdurbin joined #dataverse
