Time
S
Nick
Message
00:02 pdurbin joined #dataverse
06:49 bricas joined #dataverse
06:49 pdurbin joined #dataverse
06:49 JonathanNeal joined #dataverse
06:49 nils`` joined #dataverse
06:49 andrewSC joined #dataverse
06:49 yoh joined #dataverse
06:49 xarthisius joined #dataverse
06:49 larsks joined #dataverse
06:49 pmauduit joined #dataverse
06:49 bjonnh joined #dataverse
07:16 bricas joined #dataverse
07:16 pdurbin joined #dataverse
07:16 JonathanNeal joined #dataverse
07:16 nils`` joined #dataverse
07:16 andrewSC joined #dataverse
07:16 yoh joined #dataverse
07:16 xarthisius joined #dataverse
07:16 larsks joined #dataverse
07:16 pmauduit joined #dataverse
07:16 bjonnh joined #dataverse
08:30 jri joined #dataverse
08:32 jri_ joined #dataverse
09:50 poikilotherm joined #dataverse
10:00 poikilotherm
WOW pdurbin, your viz looks so cool and amazing!
10:00 poikilotherm
https://dev2.dataverse.org/d3/dataverse-homepage-viz/
10:07 MrK joined #dataverse
10:31 jri joined #dataverse
10:34 jri__ joined #dataverse
11:51 pdurbin
poikilotherm: I'm glad you like it! It's based heavily on https://gist.github.com/mbostock/7607535
11:53 poikilotherm
Nice :-)
11:54 pdurbin
Michael replied to https://twitter.com/philipdurbin/status/1195111126448644098 asking for technical details so maybe I'll write something up. It's a total hack right now. And still a work in progress. :)
12:15 pdurbin
Ok, I wrote up some details: https://github.com/IQSS/dataverse/issues/5603#issuecomment-554337399
12:17 poikilotherm
It would be really cool to have some usefull API endpoints for this, providing the necessary JSON
12:18 pdurbin
Yeah, that's what I was thinking. Right now it's all happening on the side with SQL and Python. Creating the JSON , I mean.
12:18 poikilotherm
Providing a possibility to retrieve a complete tree structure would be valuable for integrations, too
12:19 poikilotherm
Our research reporting staff might be actually really interested in this...
12:19 poikilotherm
Especially when they would be able to filter, too...
12:19 poikilotherm
Like "show me a nice map of all dataverses and sets that have been funded by X"
12:20 poikilotherm
On the other hand this could be done more easily on the client side
12:21 poikilotherm
But adding such metadata to the resulting JSON would be awesome, so people could actually process it
12:26 pdurbin
Yeah, for a long time I've been thinking that Dataverse should embrace its "tree-ness". Have we talked about Modified Preorder Tree Traversal (MPTT) and https://github.com/IQSS/dataverse/issues/752 ?
12:26 poikilotherm
Nope
12:29 pdurbin
There are some nice diagrams in that issue. I'd love to hear what you think. :)
12:30 poikilotherm
Interesting. I dunno if this is necessary to create that JSON you need for viz
12:31 pdurbin
Definitely not but it's an option. All the data of where every object lives in the tree is already indexed into Solr.
12:40 poikilotherm
Most likely that's gonna be a lot faster...
12:43 pdurbin
yeah
13:11 poikilotherm
Alright guys, gotta go now. Have a nice weekend :-)
13:33 pdurbin
You too.
13:38 donsizemore joined #dataverse
14:03 MrK joined #dataverse
15:23 icarito[m] joined #dataverse
15:23 rigelk joined #dataverse
16:10 donsizemore
@pdurbin bzzzt
16:13 donsizemore
@pdurbin adding harvesting logging blurbs to the harvest client page and wondering if we need to keep "New in Dataverse 4, vs. DVN 3"
16:24 pdurbin
probably not
16:27 donsizemore
cool. will send that in after lunch
17:57 pdurbin
I need some lunch too.
17:58 pdurbin
donsizemore: do you like harvesting stuff is split across two pages? Thanks for working on this.
18:08 donsizemore
@pdurbin um, I think it makes sense given the two functional perspectives
18:12 pdurbin
donsizemore: sounds good. I was just thinking that it's possible to put both pages in a "harvesting" directory kind of like this if we want: http://guides.dataverse.org/en/latest/user/data-exploration/index.html
18:13 donsizemore
@pdurbin I can do that (though I may wait until PR 6376 is accepted or rejected
18:15 donsizemore
poop, Google did away with the link: operator
18:15 pdurbin
sure, could definitely be a later thing
18:16 pdurbin
I'm just thinking that the sidebar in the Admin Guide (and the Dev Guide) is getting a little tall (a good problem to to have so many docs) and organizing them into folders might help. :)
19:43 pdurbin
donsizemore: ah ha. I just found the paper I scribbled on that had all the issues that were assigned to you.
19:46 pdurbin
There are some issues that you have self-assigned since then. :)
19:53 donsizemore
@pdurbin what's your most-important-for-me-to-complete?
19:55 pdurbin
good question, let me finishing sharing my hacky ssl cert scripts
19:57 pdurbin
those hacky scripts: https://github.com/IQSS/dataverse-ansible/issues/53#issuecomment-554505667
20:06 pdurbin
donsizemore: ok, I think I cleared the decks. You should have no issues assigned to you and nothing in the "todo" column I made for you on my board. :)
20:06 pdurbin
Is there anything you'd like to work on?
20:07 pdurbin
Because there's most important and there's somewhat fun. :)
20:07 donsizemore
@pdurbin well, the PG10 thing was SearchIT which I believe you fixed in the mean time(?)
20:07 donsizemore
@pdurbin so it might be something quick to peck off. PG10 is attractive because it's the default in CentOS8
20:08 pdurbin
Right but do we get all the devs to move to postgres 10 first?
20:08 pdurbin
Or do we let them stay on postgres 9?
20:09 pdurbin
I guess I'm trying to say it gets a little complicated quick. :)
20:09 pdurbin
Last time I think Harvard Dataverse upgraded because the version of Postgres in AWS RDS was about to go EOL.
20:10 pdurbin
So maybe we should figure out when that will happen again.
20:11 pdurbin
"We intend to support major version releases (e.g., MySQL 5.6, PostgreSQL 9.6) for at least 3 years after they are initially supported by Amazon RDS."
20:13 donsizemore
@pdurbin well, PG<10 won't work on CentOS 8 due to a psyco-pg2 library conflict
20:13 donsizemore
@pdurbin so moving from 9.6 to 10 might be you all's' call
20:14 pdurbin
yeah, that's what I'm thinking
20:15 pdurbin
That we'll be force to upgrade from pg 9 when we either move to centos 8 or when RDS declares it EOL.
20:15 pdurbin
So I think we should let that one ride.
20:15 pdurbin
even if it's easy for you
20:15 pdurbin
sorry
20:15 donsizemore
i can run the test suite one more time, see if it all passes, and close the issue
20:16 pdurbin
Sure, as the person who opened the issue, you always have the power to close it.
20:16 pdurbin
I do that a lot. :)
20:16 pdurbin
You've definitely done what you can on the Ansible side.
20:19 pdurbin
donsizemore: does https://github.com/IQSS/dataverse/issues/2911 grab you? Just testing/confirming the issue, I mean. Or https://github.com/IQSS/dataverse/issues/3215 ?
20:21 donsizemore
@pdurbin i'm happy to test it
20:22 pdurbin
thanks!
20:40 donsizemore
on PG10: SearchIT and AccessIT failed this afternoon
20:41 pdurbin
I saw a couple failures on "develop" too.
20:42 donsizemore
Internal Exception: org.postgresql.util.PSQLException: ERROR: deadlock detected
20:42 pdurbin
yikes
20:42 donsizemore
this is in EC2
20:43 pdurbin
T-T
20:45 pdurbin
donsizemore: how hard would it be to implement https://github.com/IQSS/dataverse-kubernetes/issues/85 in dataverse-ansible?
20:46 donsizemore
I might like to add some sort of marker to the various schema*.xmls used by solr these days, but it shouldn't be bad at all.
20:46 donsizemore
you're welcome to open an issue
20:53 pdurbin
Well, I was thinking we could use Oliver's new updateSchemaMDB.sh script: http://guides.dataverse.org/en/4.18/admin/metadatacustomization.html#updating-the-solr-schema
20:53 pdurbin
Load in the blocks (from a URL , like we do with public ssh keys) and then run that script, I mean.
20:54 donsizemore
works for me
20:54 donsizemore
i'm going to want to test that for our looming dataverse upgrade, anyway
21:06 pdurbin
donsizemore: cool. sshkeys with urls works fine. Just tested it. I just opened https://github.com/IQSS/dataverse-ansible/issues/133
21:32 pdurbin
Getting dark here. Time to head home. Have a good weekend, everyone!
21:35 pdurbin left #dataverse
