IQSS logo

IRC log for #dataverse, 2015-12-01

Connect via chat.dataverse.org to discuss Dataverse (dataverse.org, an open source web application for sharing, citing, analyzing, and preserving research data) with users and developers.

| Channels | #dataverse index | Today | | Search | Google Search | Plain-Text | plain, newest first | summary

All times shown according to UTC.

Time S Nick Message
00:21 searchbot joined #dataverse
00:27 pdurbin joined #dataverse
08:20 bencomp joined #dataverse
08:48 jri joined #dataverse
08:48 jri left #dataverse
09:06 romainM joined #dataverse
09:06 romainM hi
09:25 bencomp joined #dataverse
11:58 pdurbin romainM: good morning
12:58 romainM afternoon, actually
12:58 romainM :D
12:58 romainM but good morning to you ;)
13:45 pdurbin romainM: you said hi at 4 in the morning :)
13:46 romainM pdurbin: ya ya, I know it was a little bit early :D
13:47 romainM pdurbin: just jocking here ;)
13:47 pdurbin :)
13:48 pdurbin romainM: can your server reach the outside world?
13:48 romainM our dataverse server ? Yes, but there is a log entry to do so
13:49 pdurbin ok. I was wondering about "edu.ucsb.nceas.ezid.EZIDException: bad request - invalid target URL" from your logs
13:53 romainM maybe we should try an open server
13:54 romainM but the thing is that we had no problems before, with a close server so ...
13:54 romainM dunno what happened :S
13:54 pdurbin romainM: ok, that's something else I was wondering... so it used to work
13:54 pdurbin romainM: what changed?
13:54 romainM yup
13:55 romainM upgrade to the newer version of dataverse
13:55 romainM the thing is, I didn't tested the new dataverse environnement after that upgrade
13:55 romainM so I can't say if it's due to this or not
13:56 romainM cause just after this update, we opened the server with a log entry to access it
13:58 pdurbin I'm not sure what that means. "we opened the server with a log entry to access it". Do you mean you can see server.log for the first time?
14:00 romainM ah, no
14:00 romainM there is a login/password system to access our server from outside
14:01 romainM I mean, to acess the dataverse url
14:06 pdurbin ah. ok
14:16 jri joined #dataverse
14:16 romainM re jri
14:17 romainM pdurbin: jri is our system admin
14:17 pdurbin jri: hi! welcome!
14:20 jri Hi :)
14:22 romainM jri is more suitable than me if you have questions related to the serve conf, etc
14:22 romainM *server
14:23 pdurbin jri: I opened this issue yesterday because it's hard for me to know what's going wrong: https://github.com/IQSS/dataverse/issues/2780
14:25 pdurbin jri: you could try increasing debugging on a per class (or package, I believe) basis: http://guides.dataverse.org/en/4.2.2/developers/debugging.html
14:31 sekmiller joined #dataverse
14:31 pdurbin sekmiller: does "edu.ucsb.nceas.ezid.EZIDException: bad request - invalid target URL" mean anything to you? sounds bad :)
14:32 romainM hi sekmiller
14:32 sekmiller is it in your local environment?
14:32 pdurbin sekmiller: now, I'm able to publish datasets just fine. romainM and jri are having trouble
14:32 pdurbin no*
14:34 pdurbin sekmiller: it used to work for them. we aren't sure why it stopped working
14:34 jri Ok edu.harvard.iq.dataverse.api.Datasets is now to "FINE" debug level, but it don't seems to provide more info
14:35 pdurbin jri: well, that class is only exercised when you use the "datasets" API endpoint
14:35 jri Here are the last logs: http://pastebin.com/MYvWzGjY
14:35 jri Arf
14:35 pdurbin jri: maybe you could try turning up debugging on edu.ucsb.nceas.ezid
14:35 jri ok
14:35 sekmiller do you have a system property set for dataverse.siteUrl?
14:38 jri pdurbin: no more info with edu.ucsb.nceas.ezid...  Could be useful to do it on edu.harvard.iq.dvn.core.index.DOIEZIdServiceBean ?
14:38 jri or edu.harvard.iq.dataverse.DatasetPage ?
14:39 pdurbin jri: you should listen to what sekmiller is saying. he wants you to check a JVM option
14:39 sekmiller The doi service bean is where the target url is created.
14:40 sekmiller in the method getSiteUrl it forst looks for the system setting of dataverse.siteUrl
14:40 pdurbin and failing that, it looks for dataverse.fqdn it seems
14:40 jri dataverse.siteUrl should be  JVM option ?
14:42 pdurbin sekmiller: it looks to me that either dataverse.fqdn or dataverse.siteUrl needs to be configured. I guess it shouldn't hurt to configure both...
14:42 sekmiller yes
14:42 pdurbin jri: do you have either one configured?
14:43 sekmiller right.
14:45 jri I only have dataverse.fqdn in my domain.xml
14:45 jri but, it's not the fqdn
14:45 jri only the hostname of my server
14:46 pdurbin jri: can you please try making dataverse.fqdn a fqdn?
14:46 jri yep
14:51 jri Ok it's seems to work now \o/
14:51 romainM woohoo
14:51 pdurbin \o/
14:51 pdurbin sekmiller: thanks!!
14:52 romainM so ... the problem was the "fqdn" thing ?
14:52 sekmiller you're welcome.  Soemthing for the developers' guide?
14:52 jri Is it the install script that automatically put this parameter or me? don't remember...
14:53 pdurbin jri: the install script adds it
14:53 pdurbin sekmiller: jri and romainM are installing Dataverse. it should go in the Installation Guide
14:53 sekmiller OK
14:54 pdurbin jri and romainM would you be able to read and comment on https://docs.google.com/document/d/1GLky44JbtCNhGuavLXGEP8WPNdjbyHNxalEGjR78MaA/edit?usp=sharing ? I can even give you edit access if you want. We'd like to improve the Installation Guide.
14:54 romainM just a sec, not sure it works
14:56 pdurbin I just added a bit under JVM options
14:56 pdurbin sekmiller: maybe think about how the error messages could be improved
14:57 pdurbin and maybe the install script should force you to enter an fqdn? not sure
14:57 pdurbin romainM: you can't get into that Google Doc?
14:57 pdurbin it should be public and allow comments
14:58 romainM wasn't talking about that, but the "previous" bug with dataverse
14:58 romainM I can't publish actually
14:58 romainM both on internal and external
14:58 romainM :S
14:58 pdurbin then why did jri say it works now? :)
14:59 romainM it seems he did publish one thing
14:59 romainM aaaaaand, it's gone
15:00 pdurbin romainM: the problem is gone?
15:00 romainM the problem supposed to be gone is back
15:00 pdurbin d'oh!
15:00 romainM dunno how jri could publish one thing, when now I can't
15:00 romainM he can't too it seems
15:01 romainM I ... don't understand
15:02 pdurbin sekmiller: any ideas? we're tracking this issue at https://help.hmdc.harvard.edu/Ticket/Display.html?id=229931 so we can follow up there later if need be
15:02 pdurbin jri: the nuclear option is to reinstall Dataverse, I guess
15:03 sekmiller That one dataset that you published, can you clcik on the link in the citation?
15:03 romainM yup
15:03 romainM and it's not like the unpublished ones
15:04 romainM it redirect to the dataset
15:04 romainM the only one that has been published
15:04 romainM during the brief moment jri could publish
15:05 romainM as opposed to the unpublished ones that redirect here : https://www.datacite.org/testprefix
15:05 romainM when clicking on the link
15:06 romainM (the unpublished has DVN in their url, FK2 the published one)
15:06 pdurbin are you still seeing "invalid target URL" in server.log?
15:07 romainM ok, it seems it's solved
15:08 pdurbin good!
15:08 pdurbin what was it?
15:08 romainM the fqdn solved a part of the prob
15:09 pdurbin ok
15:09 romainM it's just a change in the server conf that occured this morning, when we tried something from the answer of my mail to your support
15:09 romainM update setting set content='10.5072/FK2' where name like '%Authority%';
15:09 romainM that contained DVN instead of FK2
15:09 pdurbin oh. yeah
15:09 pdurbin FK2 is for testing
15:09 romainM the published dataset stil had the good thing, so it passes ty to the fqdn
15:10 romainM (correct me if I'm bullshiting here jri)
15:10 pdurbin you should be using FK2 if you're just testing. that's what we use on our demo site. here for example: https://demo.dataverse.org/dataset.xhtml?persistentId=doi:10.5072/FK2/WPYBFR
15:12 jri Should we have to buy EZID subscription for production environment ? :/
15:13 pdurbin jri: I think you have to. sekmiller would know
15:13 romainM oh
15:13 romainM that's something we didn't knew
15:14 sekmiller for a production environment, yes you'd need to make an arrangement with EZID.
15:17 jri Well, we have to announce that to the project management team :D
15:19 romainM we may have an other doi provider
15:20 romainM dunno how it works, if we can use it, etc
15:21 romainM but anyway, thanks a lot for your help
15:22 romainM was very odd to say "hey, we installed dataverse, try it!" and then say "hey ! you can't publish"
15:22 romainM ^^
15:22 romainM at least it's working perfectly now
15:23 pdurbin romainM: I hear you. You probably don't want to publish fake/test DOIs though...
15:24 romainM given that it's a testing dataverse, that's not a prob
15:24 romainM now that I think about it
15:24 romainM I have to find the way to delete datasets in db ...
15:28 pdurbin romainM: there's a "destroy" endpoint in the datasets API
15:30 romainM the delete dataset command doesn't test if the dataset is draft or not ?
15:31 pdurbin romainM: it checks if it's published
15:32 romainM destroy is an other command then ? didn't find it in the doc yet
15:32 pdurbin romainM: it's API only
15:32 pdurbin superuser only
15:32 romainM ah
15:33 jri pdurbin or sekmiller: just to be sure, for production I need to have "dataverse.fqdn" as my real machine FQDN and "dataverse.siteUrl" as my public URL ? Our Dataverse will be behing reverse proxys and the machine name is not the public domain
15:35 pdurbin jri: those JVM options do almost the same thing. in both cases, it's for the hostname or URL you want to promote to the outside world, not the machine name
15:37 jri Ok I see... Configuring dataverse.fqdn with the public domain (that I want to promote) don't degrade other functionalities ?
15:38 pdurbin jri: nope
15:38 pdurbin go for it
15:38 jri Cool :) Thanks!
15:39 pdurbin oh sure
15:42 donsizemore joined #dataverse
15:48 pdurbin jri: were you able to get into the Google Doc?
15:48 pdurbin donsizemore: oh hi
15:49 pdurbin donsizemore: you might be interested in this: https://github.com/IQSS/dataverse/issues/952#issuecomment-161002345
15:50 donsizemore excellent =)
15:50 pdurbin bricas: it turns out fixing `vagrant up` helped another guy so thanks again
15:52 donsizemore i'm working on an ansible playbook... wondering whether it's better to stick with some minor tweaks to the installation scripts included in the release zip file, or crib from the vagrant stuff.
15:54 pdurbin donsizemore: you might want to crib from https://github.com/IQSS/dataverse-puppet
15:54 donsizemore i really like your vagrant scripts, but trying to trace where it's getting /dataverse/scripts/installer
15:55 pdurbin donsizemore: we'll, the git repo gets mounted at /dataverse
15:55 pdurbin if that helps
15:56 bencomp donsizemore: I'm interested to see your playbook(s)!
15:58 pdurbin I think https://github.com/sbgrid is using Ansible too.
15:58 donsizemore yes =) bencomp: it's pretty much spaghetti ansible from the published installation guide. i toddled along fine until i hit the "asadmin login" bit and am trying to find the right combination of cookies and ice cream to get it to store the login info
15:59 donsizemore everything works until the install script wants to restart the domain
15:59 bencomp donsizemore: is the playbook calling the shell script?
16:00 bencomp as I understand Ansible, restarting things would be a good use for handlers
16:03 donsizemore how often do your install scripts change? plan A was to have Ansible unzip dvinstall.zip and call a slightly modified script for forwards compatibility. not what you all want long-term, i know.
16:04 jri pdurbin: Yep I can access to it, I'll read and try to help when I have more time
16:11 bencomp donsizemore: if we ever make it to go 'fully Ansible' for DataverseNL, I would like to have all configuration info in our local repository and use the generic playbooks to update the service on the server(s)
16:12 donsizemore agreed. this is my first stab it at, so as you asked above i was trying to call the install script from ansible and just "make it work"
16:12 donsizemore i must run, back in a bit. thank you both for your help =)
16:12 donsizemore left #dataverse
16:13 bencomp maybe we should collaborate and ask pdurbin for the repo to hold the generic Dataverse playbooks
16:22 pdurbin jri: thanks
16:23 pdurbin bencomp: I would be more than happy to create an equivalent of https://github.com/IQSS/dataverse-puppet for Ansible and give you and others access
16:23 pdurbin may the best configuration management framework win ;)
16:25 pdurbin it would be awesome to set up tests for dataverse-puppet or dataverse-ansible: https://github.com/IQSS/dataverse-puppet/issues/10
16:26 bencomp pdurbin: I think it's time to prevent people doing the same work and create the dataverse-ansible repo. could you please create it?
16:29 pdurbin bencomp: done! https://github.com/IQSS/dataverse-ansible
16:29 bencomp pdurbin: starred! ;)
16:30 pdurbin now someone just needs to do this: git remote add origin git@github.com:IQSS/dataverse-ansible.git && git push -u origin master
16:30 pdurbin :)
17:48 pdurbin shauna: do you feel like helping me edit that job ad I showed you? http://irclog.iq.harvard.edu/dataverse/2015-11-05#i_26832
18:00 donsizemore joined #dataverse
18:00 donsizemore pdurbin: thanks for the invite =) let me get what i have a little more presentable, and i'll send you a link? could be a start if nothing else.
18:02 pdurbin donsizemore: you can just go head and push to that repo
18:03 pdurbin git remote add origin git@github.com:IQSS/dataverse-ansible.git && git push -u origin master
18:04 metamattj joined #dataverse
18:11 donsizemore pdurbin: i certainly will, but i wrote it off the top of my head over thanksgiving and just started testing it yesterday afternoon, etc. it needs a lot of work.
18:23 garnett joined #dataverse
18:23 pdurbin donsizemore: oh sure, no rush
18:24 garnett joined #dataverse
18:24 pdurbin donsizemore: so check this out: https://github.com/IQSS/dataverse/blob/4.2.2/scripts/deploy/phoenix.dataverse.org/deploy
19:03 donsizemore joined #dataverse
19:16 donsizemore pdurbin: take a peek at https://github.com/IQSS/dataverse-ansible - far from perfect, but works until mid-way through the installer script. you'd want to break that out into ansible tasks anyway, but there it is.
19:25 bencomp joined #dataverse
19:33 dataverse-user joined #dataverse
19:42 pdurbin donsizemore: nice!!! Hey, any familiarity with Travis? Can I open an issue similar to https://github.com/IQSS/dataverse-puppet/issues/10 ? :)
20:15 donsizemore joined #dataverse
20:42 axfelix joined #dataverse
20:54 bencomp donsizemore: great work on dataverse-ansible
20:55 bencomp do you keep the variables locally? and where is the role 'dataverse' defined?
21:10 jri joined #dataverse
21:18 donsizemore we'd want to add an example group_vars file (mine is outside the git tree)
21:18 donsizemore ansible will automatically select tasks/main.yaml; i used includes to split up the prereq and install bit.
21:19 donsizemore phil sent me his phoenix stuff so i'm reading through it and matching up variables
21:20 donsizemore mine is also centos7-specific (i was writing it to cobble it into the iPlant Discovery Environment playbooks) so we'll need to add distro checks to play nice with Debian
21:23 pdurbin donsizemore: ah. so maybe your Ansible stuff wouldn't play nicely with Travis since you're targeting centos (which makes sense) and Travis uses ubuntu.
21:28 donsizemore pdurbin: it could pretty easily be made OS-agnostic with ansible_distribution checks
21:28 bencomp donsizemore: I've been thinking a bit, and I'm not sure yet, that perhaps this repo should be a 'pure' role that can be included in a site.yml and galaxy. it could have a defaults folder
21:29 pdurbin donsizemore: cool. what I want (maybe not explained well enough in that issue) is automated integration tests on Travis like this: https://travis-ci.org/openscholar/openscholar/jobs/94158615
21:33 pdurbin basically, Travis spins up an Ubuntu VM, installs Apache, Solr, etc.
21:33 pdurbin Ansible playbook for Dataverse - Google Groups - https://groups.google.com/forum/#!topic/dataverse-community/VsoDw71PfBI
21:33 pdurbin :)
21:47 donsizemore bencomp: definitely. i just offered what i had as a starting point
21:54 bencomp donsizemore: do you mind if I create a short readme?
22:08 pdurbin donsizemore bencomp: hey I just added https://github.com/pameyer (from https://github.com/sbgrid ) to https://github.com/orgs/IQSS/teams/dataverse-ansible
22:17 bencomp great. I just added a README :)
23:29 metamattj joined #dataverse
23:55 pdurbin +1

| Channels | #dataverse index | Today | | Search | Google Search | Plain-Text | plain, newest first | summary

Connect via chat.dataverse.org to discuss Dataverse (dataverse.org, an open source web application for sharing, citing, analyzing, and preserving research data) with users and developers.