IQSS logo

IRC log for #dataverse, 2015-08-14

Connect via chat.dataverse.org to discuss Dataverse (dataverse.org, an open source web application for sharing, citing, analyzing, and preserving research data) with users and developers.

| Channels | #dataverse index | Today | | Search | Google Search | Plain-Text | plain, newest first | summary

All times shown according to UTC.

Time S Nick Message
04:33 michbarsinai joined #dataverse
07:56 bencomp joined #dataverse
11:25 pdurbin bencomp: did you see that wiki page? https://wiki.archivematica.org/Dataverse
11:25 bencomp pdurbin: yes
11:26 bencomp I remember thinking of updating it
11:26 bencomp no, wait, that was not this page
11:26 bencomp I hadn't seen this one
11:27 pdurbin pretty sure it's new
11:27 bencomp pdurbin: I had thought of updating http://coptr.digipres.org/Dataverse
11:28 pdurbin interesting... Community Owned digital Preservation Tool Registry (COPTR)
11:29 pdurbin bencomp: just doing s/DataVerse/dataverse/ would help a lot :)
11:29 bencomp heh
11:32 bencomp pdurbin: see how Artefactual first publicly works on requirements before they start implementation? ;)
11:47 bencomp okay, that was harsh - sorry
12:09 pdurbin bencomp: this week we sent this email: Improving the organization + display of files in Dataverse - Google Groups - https://groups.google.com/forum/#!topic/dataverse-community/gtz2npccWjU . Does that help? :)
12:20 bencomp pdurbin: yes! :) (even though the FRD is pretty big)
12:22 pdurbin it's a big feature :/
12:31 bencomp commented
12:32 pdurbin bencomp: where?
12:39 bencomp on the doc, under "use case 1"
12:39 bencomp s/on/in/
12:40 pdurbin so many docs
12:40 pdurbin oh oh, I see it
12:41 pdurbin bencomp: what you wrote makes sense
12:42 pdurbin on a related note, I'm not sure I should be calling this a "user story" https://docs.google.com/document/d/1FzgFkjyZH4NL8Q-6Ytl3l3xWib5qSwH2jhud4H6yM94/edit?usp=sharing
12:42 pdurbin but I'm not sure what a good name for it would be
12:43 pdurbin in the end it will boil down to lines like assertEquals(expectedFiles, actualFiles) https://github.com/IQSS/dataverse/commit/fa52632
12:47 bencomp looks like a test plan or something?
12:48 pdurbin ah, test plan
12:48 bencomp I imagine a user story is a sequence of use cases to achieve something
12:49 pdurbin so a test plan is for a single test?
12:49 bencomp I see you wrote scenario - that covers it maybe even better
12:50 pdurbin basically I'm trying to write in words what I will later code up as a test
12:50 bencomp I guess you can have abstract test plans and implementations of those abstract plans
12:51 pdurbin maybe "scenario" is good
12:52 bencomp I don't have strict definitions of these terms right here right know
12:52 bencomp yes, scenario
12:52 pdurbin I think a user story is less detailed.
12:52 bencomp also helps describing bugs, as you'll know :)
12:53 bencomp "I did X, Y, Z and expected A, but got B"
12:53 pdurbin sure. reproducibility
12:55 bencomp in the context "User story for using Solr for file listing on dataset page", "user story" is definitely "wrong", as I don't expect a user to care for the implementation details :)
12:57 pdurbin so maybe "scenarios" and there could be multiple (which would become separate tests)
13:00 pdurbin really these would be acceptance tests
13:02 bencomp which you can implement as integration tests, or leave as a test plan for humans to use
13:04 pdurbin oh sure
13:04 pdurbin but I'd like to automate testing if I can
13:04 pdurbin not all testing
13:04 pdurbin I'd like to automate some of the testing, I mean
13:06 bencomp I'd like you to automate as much testing as possible :)
13:07 pdurbin bencomp: well, you can help me define the scenario(s) if you want :)
13:14 bencomp pdurbin: I think you can expect one, maybe more, today
13:15 pdurbin cool
13:16 pdurbin what's nice is that this is forcing me to write methods in rest-assured (Java) that previously I only had bash scripts for. to test file deletion via SWORD, for example
13:17 bencomp did you know that my "OO software engineering" book explains that use cases are defined as interaction scenarios between user and system?
13:18 pdurbin sounds like a nice book
13:19 bencomp and use cases can include or extend other use cases, or be a variant of other use cases
13:20 pdurbin turtles all the way down
13:20 bencomp hehe
13:22 pdurbin bencomp: I should get you writing rest-assured tests. it's fun!
13:23 bencomp only it's not my job and I decided to stop interfering with Dataverse code in my own time
13:24 bencomp pdurbin: or are you offering me a job? :D
13:24 pdurbin heh
13:25 pdurbin bencomp: well, if you change your mind, here's a nice small one I started recently: https://github.com/IQSS/dataverse/blob/e45dd7939a6af0e997b672dc3e1c8446ea6d0f31/src/test/java/edu/harvard/iq/dataverse/api/BuiltinUsersIT.java
13:25 pdurbin bencomp: and... how can I make my rest-assured tests count toward my code coverage. currently they don't. :( T-T
13:27 bencomp I have no idea
13:27 bencomp the interwebs should have something on this
13:28 pdurbin nothing in code or issues at https://github.com/jayway/rest-assured/search?utf8=%E2%9C%93&q=jacoco
13:28 bencomp an IT does not call the code directly of course
13:29 pdurbin yeah
13:29 pdurbin jacoco is mentioned here at least: https://groups.google.com/d/msg/rest-assured/rzDLiQVAtzY/O11u4ZCS4z0J
13:30 bencomp in ehri-project.eu, we used a somewhat different approach
13:32 bencomp in https://github.com/EHRI/ehri-rest/blob/master/ehri-importers/src/test/java/eu/ehri/project/importers/AbstractImporterTest.java we defined an abstract unit test that sets up a surrogate Neo4J graph, loads it with sample data ("fixture data")
13:33 bencomp I never checked the code coverage for that code
13:34 pdurbin bencomp: you should. how much coverage is there?
13:35 bencomp but by putting (potentially) problematic data in the fixture data and trying to process (import, in my case) it correctly in unit tests, I don't think we had real problems putting stuff in production
13:38 bencomp pdurbin: does jacoco need to be in the pom for the netbeans integration to work?
13:39 pdurbin probably
13:39 pdurbin it's in the pom now. in the 4.2 branch
14:22 axfelix joined #dataverse
14:39 buckwheat joined #dataverse
14:42 buckwheat Morning All. Almost have this thing running, for what ever reason I cannot seem to get it to create/set/accept a root dataverse. The primary issue seems to be "A system exception occurred during an invocation on EJB DataverseServiceBean, method: public edu.harvard.iq.dataverse.Dataverse edu.harvard.iq.dataverse.Dataverse‚ÄčServiceBean.findRootDataverse()]]" and it appears to be caused by "Caused by: javax.persistence.NoResultException: getSi
14:43 pdurbin buckwheat: hi! well, that's the expected error when you go to http://localhost:8080 if you haven't created a root dataverse via the API yet.
14:44 pdurbin basically, the homepage is looking for the root dataverse
14:45 buckwheat I have run setup-all three times now. this last time it appeared to complete without any errors....nuts
14:46 pdurbin buckwheat: you could look in the database to see if the root dataverse was created
14:47 buckwheat Is that where it is stored? looking now, without a schema may take a moment to find the right table
14:47 pdurbin there's a "dataverse" table: https://apitest.dataverse.org/guides/developers/database/schemaspy/tables/dataverse.html
14:47 buckwheat thanks
14:47 pdurbin sure. see also all the relationships: https://apitest.dataverse.org/guides/developers/database/schemaspy/relationships.html
14:48 pdurbin buckwheat: is there a reason why you are running "setup all" manually instead of letting the Perl script ("install") run it?
14:50 buckwheat yes, when we chatted yesterday you mentioned thats what set the root dv and I could/should run that. What is the value I need to set for the root dataverse in the table?
14:55 pdurbin buckwheat: you use some JSON to create the root dataverse
14:56 pdurbin it looks like this: https://github.com/IQSS/dataverse/blob/master/scripts/api/data/dv-root.json
14:57 buckwheat thanks
14:57 pdurbin http://guides.dataverse.org/en/latest/installation/installation-main.html talks about the JSON file for creating the dataverseAdmin user. It's similar.
14:57 pdurbin buckwheat: anyway, the idea is that the Perl script should do all this for you
15:07 buckwheat pdurbin: pdurbin: believe me, wish it was doing that. whats odd is everything else seems to take, but what seems to be the most crucial bit of info to start this beast refuses to create/set.
15:08 donsizemore joined #dataverse
15:09 buckwheat pdurbin: you available for a quick phone call? Love to get this running :)
15:12 buckwheat pdurbin: just got pulled into a meeting so have to pick this back up later. Thanks for being helpful, much appreciated.
15:18 buckwheat joined #dataverse
15:25 bencomp pdurbin: took a while, but the ehri-importers module has 80.32% test coverage
15:25 pdurbin bencomp: very good!
15:29 bencomp and the "Core API" has 78.42%
15:41 pdurbin nice
15:48 axfelix joined #dataverse
17:23 axfelix joined #dataverse
17:33 cnk joined #dataverse
17:54 axfelix joined #dataverse
18:55 donsizemore joined #dataverse
19:05 buckwheat joined #dataverse
19:10 buckwheat Hey pdurbin, got our Dataverse up and running. After fighting 4.1 for a day, dropped the db, refreshed solr and installed 4.0.1. Went perfectly. Going to give our users a few days to get to know the software then will try the 4.1 upgrade. Thanks for your time, much appreciated.
19:17 pdurbin buckwheat: I'm glad you got it working!
19:18 pdurbin so sorry to hear about all the trouble
19:18 pdurbin buckwheat: I assume you still don't know why 4.1 wasn't easy to install
19:18 pdurbin but I'm glad 4.0.1 worked for you
19:23 buckwheat it happens, no worries. Will say that if the perl script would sanity check along the way and stop/die when key items like the root dataverse dont take would be a big help. I really dont know why 4.1 bombed and 4.0.1 sailed thru.
19:24 pdurbin doesn't make any sense to me
19:24 buckwheat pdurbin: I used the same db, same solr, same glassfish...just wiped out configs, dropped db's and ran install
19:25 buckwheat pdurbin: will keep a close eye on the 4.1 UPGRADE and let you know how it goes. Thanks again for fielding questions out of the blue. Most helpful
19:26 pdurbin buckwheat: I don't *think* you ran into this: https://github.com/IQSS/dataverse/issues/1749 . We didn't make major changes to the Solr schema between 4.0.1 and 4.1.
19:27 pdurbin buckwheat: here are my notes on upgrading a server from 4.0.1 to 4.1. Might be helpful: http://irclog.iq.harvard.edu/dataverse/2015-08-12#i_22898
19:31 buckwheat pdurbin: Thanks, our institution is going big on Dataverse so we systems guys want to make sure we understand whats going on under the hood. Also, Im getting old so documentations doubles as memory for me ;)
19:32 pdurbin buckwheat: did you see that a systems guy in the Netherlands is working on a Puppet module?
19:33 pdurbin https://github.com/IQSS/dataverse/pull/2407
19:33 buckwheat I did, we use puppet too!
19:33 pdurbin he's on holiday all month I think
19:35 buckwheat pdurbin: have a great weekend and thank you for taking time for questions.
19:35 pdurbin you too
19:47 donsizemore joined #dataverse
19:48 bencomp joined #dataverse
19:50 axfelix joined #dataverse
19:59 axfelix joined #dataverse
20:55 axfelix joined #dataverse
21:08 bencomp pdurbin: https://docs.google.com/document/d/1PnbRDKj6XHpUmwsfCApfV8NrSwFfosomEggerIHQvzU/edit?usp=sharing a test scenario
21:09 bencomp it's still today :)
22:05 axfelix joined #dataverse
22:06 axfelix joined #dataverse
22:13 axfelix joined #dataverse
22:58 axfelix joined #dataverse
23:02 pdurbin bencomp: thanks!

| Channels | #dataverse index | Today | | Search | Google Search | Plain-Text | plain, newest first | summary

Connect via chat.dataverse.org to discuss Dataverse (dataverse.org, an open source web application for sharing, citing, analyzing, and preserving research data) with users and developers.