Time
S
Nick
Message
00:45 garnett joined #dataverse
01:10 garnett joined #dataverse
01:47 garnett joined #dataverse
04:31 garnett joined #dataverse
05:18 axfelix joined #dataverse
08:29 jri joined #dataverse
11:25 jri_ joined #dataverse
11:30 jri joined #dataverse
11:33 jri_ joined #dataverse
11:34 jri joined #dataverse
11:46 jri joined #dataverse
12:20 telnoratti joined #dataverse
12:44 majest1c joined #dataverse
12:44 majest1c joined #dataverse
13:26 donsizemore joined #dataverse
13:47 rhiaro_ joined #dataverse
14:00 pdurbin
Happy New Year, everybody. :)
14:37 jri joined #dataverse
14:41 pameyer joined #dataverse
15:04 pdurbin
balo bjonnh bret bricas_ donsizemore jeffspies______ jhand JonathanNeal jri karissa majest1c pameyer petrichor rhiaro sivoais skay telnoratti: community call in 2 hours if anyone would like to call in: http://dataverse.org/community-calls
15:04 balo
happy new year!
15:04 JonathanNeal
Happy New Year
15:05 skay
hi, happy new year
15:05 skay
:)
15:07 pameyer
happy new year
15:21 jri joined #dataverse
15:26 djbrooke joined #dataverse
15:28 bjonnh
pdurbin: sorry I can't have access to that today :(
15:31 pdurbin
bjonnh: no worries
16:00 yoh joined #dataverse
16:27 pdurbin
pameyer: I'm going to resume work on https://github.com/IQSS/dataverse/issues/3352
16:27 pameyer
pdurbin: sounds good
16:39 djbrooke joined #dataverse
16:48 djbrooke joined #dataverse
16:50 andrewSC joined #dataverse
16:50 andrewSC
hi all
16:50 djbrooke joined #dataverse
16:50 pameyer
hi
16:51 andrewSC
I was just asked about exporting results of a search from dataverse into a csv or similar format... It doesn't look like dataverse has this functionality in the ui?
16:52 pameyer
I don't think so
16:52 pameyer
pdurbin: you probably have a better idea, what do you think?
16:53 andrewSC
kk no worries, I was just wondering before I wrote some sort of script to do that
16:54 pameyer
you could define a harvesting set based on the search, and get xml out that way
16:54 pdurbin
andrewSC: this isn't quite what you're asking for, but have you played with the "as_csv" stuff at https://services.dataverse.harvard.edu/static/swagger-ui/index.html?url=/miniverse/metrics/v1/swagger.yaml#!/metrics_-_datasets/get_datasets_file_stats ?
16:54 pameyer
not sure if that would be helpful in your case or not
16:55 andrewSC
pameyer: no I haven't but I'll certainly take a look now
16:56 andrewSC
hmmmm yeah I think my use case is a little more esoteric than this
16:56 pameyer
more flexibility in the search parameters?
16:57 andrewSC
basically i have a bunch of datasets that have been tagged with keywords, but the keywords semantically form a hierarchy
16:57 djbrooke joined #dataverse
16:57 andrewSC
i basically need to re-generate this hierarchy in xml so you can expand/hide groups
16:57 andrewSC
with some sort of xml viewer
16:57 andrewSC
does that make sense?
16:58 pameyer
it does - sounds like it would need some scripting in any case
16:58 andrewSC
yeah... lol
16:59 pameyer
I've had *reasonable* luck with xmllint ;)
16:59 andrewSC
hahah
16:59 andrewSC
yeah we'll see what I end up coming up with...
17:00 pameyer
good luck - sounds interesting
17:00 andrewSC
thankfully management came to me *early* with the concern rather than last minute :)
17:00 andrewSC
ty
17:05 sekmiller joined #dataverse
17:13 andrewSC
BORLAND DELPHI
17:13 andrewSC
whoops wrong channel, sorry lol
17:37 pameyer
borland delphi might be able to do xml :)
17:38 donsizemore joined #dataverse
17:39 djbrooke joined #dataverse
17:48 pdurbin
andrewSC: your keywords form a hierarchy. Have you already published your dataset? If not, want to create one on https://demo.dataverse.org ?
17:49 andrewSC
pdurbin: I was actually just reading https://github.com/IQSS/dataverse/blob/v4.2.1/scripts/issues/2649/reproduce because the datasets aren't published... (draft only)
17:49 andrewSC
and sure let me give it a go
18:01 djbrooke joined #dataverse
18:08 andrewSC
pdurbin: https://demo.dataverse.org/dataverse.xhtml?alias=hierarchy
18:09 andrewSC
like I was saying it's just a tree
18:09 andrewSC
so for example I'd just want all entries with "X1" as a tag then grouping by "P*" then "A*"
18:09 andrewSC
where P* and A* all have X1 as a parent
18:10 andrewSC
does that make sense?
18:16 djbrooke joined #dataverse
18:20 andrewSC
would this get me what I want? https://github.com/IQSS/dataverse-client-python/blob/master/dataverse/utils.py#L17
18:20 pdurbin
andrewSC: sort of but I'm multitasking and about to go to lunch. You know you can facet on "X1", right? In the UI it looks like this but you can do the same from the Search API : https://demo.dataverse.org/dataverse/hierarchy?q=&fq0=keywordValue_ss%3A%22X1%22
18:21 andrewSC
pdurbin: kk no worries, I'll crank on this and once you're a little more free feel free to ping me :)
18:21 pdurbin
andrewSC: ok, meanwhile, please also take a look at https://github.com/IQSS/miniverse
18:21 andrewSC
mmmmmm interesting :)
18:31 djbrooke joined #dataverse
19:04 andrewSC
also
19:04 andrewSC
https://gist.github.com/andrewSC/7d22f039f9c8b7d62952b4705a2ec6c6
19:04 andrewSC
while I know this is undocumented (and most likely not supported) I noticed when i set show_facets to true I still don't see facets in the output?
19:05 andrewSC
and this is the unpublished dataset https://cl.ly/013623081R41/Screen%20Shot%202017-01-03%20at%202.05.11%20PM.png
19:06 andrewSC
also noticed locally that specifying a facet for draft/unpublished datasets still returns all datasets?
19:36 telnoratti joined #dataverse
19:50 pdurbin
BORLAND DELPHI
19:50 andrewSC
yes
19:50 pdurbin
I'm back. :)
19:50 andrewSC
lol
19:50 andrewSC
nice
19:51 andrewSC
yeah i'm kinda stuck tbh
19:51 pdurbin
andrewSC: show_facets is for the *search* api, not that undocumented mydata api
19:51 andrewSC
i think I'm at the point where I'll need to query solr directly?
19:51 andrewSC
ahh gotcha
19:52 pdurbin
andrewSC: do you have access to Solr? You only do if you run your own Dataverse installation.
19:52 andrewSC
i do
19:53 andrewSC
i setup glassfish and all that lol
19:53 pdurbin
oh! great! that opens up a lot of possibilities
19:53 andrewSC
I was also the guy in here a while back that removed DOI's from the webviews for my org
19:53 pdurbin
I mean, would a SQL query be easier?
19:53 andrewSC
at this point it very well might be
19:53 pdurbin
so that "miniverse" app might help you
19:54 andrewSC
mmmmmmmm
19:54 * andrewSC
looks again
19:54 pdurbin
it's a python app with read only access to a Dataverse database
19:55 andrewSC
I'm dying
19:55 andrewSC
my install is 4.3.1
19:55 andrewSC
lmao
19:55 andrewSC
all pain no gain
19:56 pdurbin
you must suffer for your data
19:56 andrewSC
hahah
19:57 andrewSC
so does this mean i need to upgrade my DV install before I install miniverse?
19:57 pdurbin
yeah
19:57 andrewSC
given the large print that says 4.4+
19:57 andrewSC
yeah
19:57 andrewSC
kk
19:57 pdurbin
or query Solr directly
19:57 pdurbin
up to you :)
19:57 andrewSC
lol
20:01 andrewSC
pdurbin: so after i install miniverse there's some sort of sql frontend? I just looked at the miniverse at https://services.dataverse.harvard.edu/
20:02 andrewSC
oh nvm
20:02 andrewSC
there's still coding involved using the django orm
20:03 pdurbin
andrewSC: hmm, no there is no SQL front end
20:03 pdurbin
miniverse has a bunch of SQL queries defined
20:04 andrewSC
there was actually a pretty interesting project i came across the otherday
20:05 andrewSC
https://github.com/groveco/django-sql-explorer
20:06 axfelix joined #dataverse
20:09 pdurbin
andrewSC: cool, the tool I've got my eye on is Metabase, which I mentioned here: https://github.com/IQSS/dataverse/issues/3099
20:09 andrewSC
yaaaa I came across this guy!
20:12 andrewSC
I'll have to try this
20:12 pdurbin
it seems neat, and it's written in Clojure, which is fun :)
20:13 andrewSC
;)
20:13 andrewSC
lol
20:13 andrewSC
I haven't played with it yet tbh
20:14 andrewSC
TIL there's a walmart labs http://www.walmartlabs.com/
20:14 pdurbin
andrewSC: why XML anyway? What are you building?
20:15 andrewSC
pdurbin: that wasn't a requirement, just a random thought realizing i can just as well use json... lol
20:15 pdurbin
oh, ok
20:15 andrewSC
for whatever reason i made the connection with xml<->trees
20:15 pdurbin
yeah, xml is good for trees
20:16 pdurbin
pameyer: should we revisit https://github.com/sbgrid/data-capture-module/issues/10 ? I'm about to run `vagrant up`.
20:17 pdurbin
whoa! "docker create" too? huh, maybe I should use docker instead?
20:19 andrewSC
docker all the things
20:19 pameyer
pdurbin: docker isn't extensively tested; that was partly an exercise of learning docker from a meeting
20:22 pdurbin
ok, I guess I'll stick with Vagrant then. what about that issue 10?
20:26 pameyer
you should've hit all the undocumented parts last time around
20:26 pameyer
but it'll probably need another ansible run for the rq parts if you're re-using an old vm
20:26 pdurbin
so after installation I should edit lighttpd.conf and restart lighttpd?
20:30 andrewSC
Alright folkssss gonna sign off, I'll be back on tomorrow though :) have a good afternoon! Thanks!
20:31 pameyer
pdurbin: it *should* be in the ansible variables
20:32 pameyer
but yeah
20:34 pameyer
and vars/secrets.yml.template should be usable as a template for secrets.yml
20:34 pdurbin
yeah, I just opened https://github.com/sbgrid/data-capture-module/issues/14
20:35 pameyer
yup - I saw
20:35 pameyer
(doh) - hadn't read
20:59 axfelix joined #dataverse
21:00 pdurbin
pameyer: oh, speaking of the DCM, Ryan from TDL called in earlier and *may* have one of his people kick the tires on the DCM. He's also interested in Cloud Dataverse. I think we still want DCM beta testers.
21:03 pdurbin
this TDL: https://dataverse.tdl.org
21:20 pameyer
pdurbin: good to hear
21:21 pameyer
hopefully it'll be another boost for me to improve the docs
21:23 pdurbin
I'm adding better error handling on the Dataverse side if the DCM returns 403/forbidden.
21:24 pdurbin
We should know next week if they'll be kicking the tires any time soon or not.
21:39 pameyer
with other stuff going on, I probably won't get a chance to improve the docs this week
21:42 axfelix joined #dataverse
22:53 pdurbin
that's fine, no rush
23:27 djbrooke joined #dataverse
23:47 pdurbin
here are the notes from the call: https://docs.google.com/document/d/1vm9jT9DKKqudaekK1fztqbcPxzogrd840w92fdBC4Zk/edit
23:49 djbrooke joined #dataverse
