IQSS logo

IRC log for #dataverse, 2018-11-07

Connect via chat.dataverse.org to discuss Dataverse (dataverse.org, an open source web application for sharing, citing, analyzing, and preserving research data) with users and developers.

| Channels | #dataverse index | Today | | Search | Google Search | Plain-Text | plain, newest first | summary

All times shown according to UTC.

Time S Nick Message
00:26 jri joined #dataverse
04:27 jri joined #dataverse
05:52 nanoz joined #dataverse
08:06 jri joined #dataverse
08:23 MrK joined #dataverse
08:39 MrK Hi, me and me friend wanna work on some new features and I was wondering if we could be added to the contributor list and our issue assign to us?
09:26 poikilotherm joined #dataverse
10:26 nanozz joined #dataverse
11:06 pdurbin MrK: hi! I'm looking through issues. How do you feel about this one? https://github.com/IQSS/dataverse/issues/5137
11:08 poikilotherm Morning pdurbin :-)
11:08 poikilotherm Sorry I couldn't make it to the community call yesterday (evening for me)
11:10 pdurbin poikilotherm: no problem. During tech hours we talked about deps and payara.
11:13 poikilotherm Oh cool :-) Did you guys came up with something or still discussing things?
11:25 msy_inist joined #dataverse
11:31 pdurbin poikilotherm: Gustavo and Matthew are going to talk more about which application server to target.
11:34 poikilotherm Ok
11:34 poikilotherm Then I'll wait for whatever comes up from that...
11:34 poikilotherm In the mean time, I'm starting to get my hands dirty with dependencies...
11:35 poikilotherm There is plenty of room for housekeeping business...
11:35 poikilotherm Should I look more into that or what was the result of tech hours for that?
11:40 pdurbin I guess people were wondering how you were planning to keep dependencies up to date when you put them in the a "lib" directory in Glassfish.
11:41 poikilotherm Everytime the container is rebuilt, the deps are updated from what is defined within the maven project
11:41 poikilotherm Then the new container image needs to rolled out on docker/kubernetes and of course containers need to restart
11:42 poikilotherm But that's the hole thing about the uber/fat/hollow jar topic...
11:43 poikilotherm I kinda ship around that by sticking with the WAR concept of deployments, but adding the dependencies on another level
11:44 poikilotherm By that you still need the application server. Getting rid of that is even more to the fat/uberjar side
11:44 poikilotherm (Because you actually package both deps AND app server within the JAR)
11:44 pdurbin Sure, there are multiple dimensions here.
11:45 pdurbin The feeling in the room was that we're happy for you to experiment with stuff.
11:45 poikilotherm The nice thing is: by sticking with the WAR concept, the maven build is not that much different for container and "classic mode"
11:45 poikilotherm Ok
11:47 poikilotherm Then I will stick with fixing the dependency stuff for AWS by using BOMs for AWS and Jackson and stop there, right? Or shall I continue with some more dependency convergence conflicts?
12:09 MrK pdurbin: Oh sry I wasn't precise, we are from the branch of University of Warsaw and we wanted to commit to the project in order to use it for our own requirements. Since dataverse has quite a lot of features we thought that why not just join and add the features we need, since it will benefit everyone. Would it be possible to assign the issue to the creator? https://github.com/IQSS/dataverse/issues/5280.
12:14 pdurbin MrK: oh! Great! So you have a specific feature in mind. That's fine but our design team would probably want to give their opinion on that feature. I wouldn't want you to spend hours and hours coding it up and then hear that it's not what they want.
12:15 pdurbin MrK: the design team is currently working on this issue that relates to the one you opened: https://github.com/IQSS/dataverse/issues/3404
12:16 pdurbin MrK: so I would suggest leaving a comment on #3403 introducing yourself and linking to your suggestion at #5280
12:16 pdurbin MrK: I hope that makes sense. I don't want to discourage you! I can try to remember to mention you during the design meeting this afternoon.
12:18 pdurbin poikilotherm: I'm kind of thinking you should talk to Gustavo and Matthew, to be honest. Again, our main interest is not being stuck on an a specific version of Glassfish from years ago.
12:20 pdurbin MrK: I should add that users could really use a way to organize their files so I'm glad you're thinking about this!
12:21 MrK pdurbin: ah no worries we are doing research how the process looks so it's good to know, I'm also gonna have to talk to our decision makers if they are fine with it since in the future there are going to be deadlines :D. I was also wondering if ofc it's even possible to predict, how much time does it usually take to review the new feature we would like to add?
12:23 pdurbin Well, it really depends on the feature. Generally speaking, API only features are less controversial. Features that don't affect the UI or UX, I mean.
12:23 pdurbin MrK: Do you have ideas for other features?
12:24 pdurbin It's kind of sad but sometimes users resort to naming their files 0foo.txt and 1.bar.txt to keep them in the order they want. :/
12:27 MrK pdurbin: Right now we have one more which is a bit more controversial, it's pretty much an ability to apply additional CC licenses.
12:28 pdurbin MrK: funny you should mention that because there's been some chatter about that in Slack within the last week or so.
12:29 pdurbin It's less controversial than you think. I could probably dig up some recent mockups with multiple CC license options.
12:30 poikilotherm pdurbin: do they usually hang around on your slack channel? willing to go over there and talk to them, but most certainly I will need an account.
12:31 pdurbin poikilotherm: sure, the #dv-design channel is quite active
12:31 MrK oh interesting, that's great we have the plans how it should look, so it would be great to coordinate and chat about details
12:31 pdurbin MrK: can you see the Google doc linked from the most recent comment at https://github.com/IQSS/dataverse/issues/1990 ?
12:32 poikilotherm What do I need to get inside that channel? Google doesn't spit out much about where and how to get access...
12:32 MrK pdurbin: oh so on this slack channel there are people from the design team?
12:33 pdurbin MrK: right, they have their own channel
12:33 poikilotherm Drop a mail in case you don't want that to be logged publically
12:33 poikilotherm ;-)
12:34 MrK pdurbin: sadly I need to reques the access in order to view the file
12:34 pdurbin MrK: ok, I would suggest leaving a comment on the issue asking if that Google doc can be opened up.
12:36 pdurbin poikilotherm: I don't know what the process is for people joining Slack but I guess you could try emailing support@dataverse.org about this. My understanding is that we pay per person per month for accounts. I like IRC because it's free. :) And based on an RFC. :)
12:37 pdurbin MrK: leaving a comment on https://github.com/IQSS/dataverse/issues/1753 would be good too.
12:37 poikilotherm You are aware about free plans for OSS?
12:37 pdurbin poikilotherm: buh, I'm more of an IRC guy
12:38 poikilotherm *G* I like the capabilities like image/site preview, markdown support etc.
12:38 poikilotherm And there is a IRC bridge IIRC
12:38 pdurbin poikilotherm: no, Slack recently killed their IRC and XMPP gateways
12:38 poikilotherm But I'm fine with both
12:38 pdurbin poikilotherm: how do you feel about Gitter?
12:38 poikilotherm Oh sad to hear that
12:39 poikilotherm Gitter is far from usable. Used it for a project. Unstable UI, unusable apps losing messages and notifications, overall a very unpleasant XP
12:40 poikilotherm Mattermost and Rocketchat seem to be a good alternative to Slack
12:40 poikilotherm We use Rocketchat here at FZJ and it keeps getting attraction
12:40 pdurbin poikilotherm: huh, ok. Gitter works fine from its IRC gateway for me. And I recently installed the Android app and it seems to work well enough. Gitter is free and open source and good for transparency in open source, which Slack is not.
12:41 poikilotherm Sure :-) That's why we choose Rocketchat ;-)
12:41 pdurbin Can Rocketchat have public logs like Gitter?
12:42 * poikilotherm reads docs
12:43 pdurbin The nice thing about Gitter is that public logs are the default. It's part of the culture of using Gitter.
12:44 poikilotherm You can create public channels in Rocketchat and give write permissions to just a bunch of people
12:45 MrK what's the slack adress for dataverse?
12:45 pdurbin poikilotherm: cool. Can you create one so I can see how it looks?
12:45 poikilotherm Most likely the broadcast channels should be a good match
12:45 poikilotherm https://rocket.chat/docs/user-guides/channels/#creating-channels
12:45 poikilotherm Sorry - our Rocketchat is on the intranet only due to regulation issues like GPDR etc
12:45 poikilotherm Will take time to solve that
12:46 poikilotherm Oh and I am not sure if you can see the channel without a user account
12:46 poikilotherm But of course you can hook it to something like Github and the like
12:50 poikilotherm Riot looks promising too - world readable chat is a baked in feature
12:51 poikilotherm Oh that's based on Matrix...
12:51 poikilotherm https://matrix.org
12:51 pdurbin MrK: https://iqss.slack.com
12:52 pdurbin poikilotherm: does RocketChat have an IRC gateway?
12:53 poikilotherm IIRC yes
12:54 jri joined #dataverse
12:54 pdurbin cool
12:55 poikilotherm Jepp, there are reports about people using it, but it is still a "work in progress"
12:56 pdurbin I'm bummed you don't like Gitter.
12:57 MrK pdurbin: is it possible to join the slack? In sign up form I see you need an harvard email. Also I had a question about changing schema, if you guys change anything in schema, you are just including it in upgrade sql and release it with new version?
12:59 poikilotherm I had some bad experience back in April with it... I would be glad to try again...
12:59 pdurbin MrK: you could try asking at support@dataverse.org
12:59 poikilotherm pdurbin: just did that... ;-)
12:59 pdurbin poikilotherm: Gitter is hiring. Maybe it'll be better in the future. :)
13:04 jri joined #dataverse
13:16 poikilotherm pdurbin: current outcome of my research about the maven dependency problems... https://github.com/poikilotherm/dataverse/commit/0b19800fd699fe8159478629c7b9da2170b3e5b4
13:19 poikilotherm Maybe I should a word or two about dependency management to the dev docs, too
13:49 poikilotherm +write
13:49 poikilotherm Something is wrong in my head today. Skipping words...
13:54 isullivan joined #dataverse
14:02 donsizemore joined #dataverse
14:16 poikilotherm Hey pdurbin... Just found an excellent case of why that dependency management is important... Moved from JaCoCo 0.7.5... to 0.8.2 and removed it from the direct dependencies. That lead to errors about missing libraries for Dataverse code, that has been available via the transitive dependency before...
14:20 poikilotherm Whoops. You relied on edu.emory.mathcs.backport.java.util.Collections within GitHubOAuth2AP.java and util.json.JsonPrinter
14:21 poikilotherm That most certainly was by accident
14:24 poikilotherm And I fixed two places using org.codehaus.plexus.util.StringUtils instead of org.apache.commons.lang.StringUtils
14:24 poikilotherm Most certainly these two packages have been around because of the direct dependency to JaCoCo
14:25 poikilotherm And then during coding the IDE offered it as a quick import
14:35 nanoz joined #dataverse
14:45 pdurbin poikilotherm: yeah Netbeans keeps suggesting that package from emory. It's annoying. I don't know if we even use it.
14:46 poikilotherm I hope Netbeans only will offer it because of the former direct JaCoCo dependency
14:47 poikilotherm At least IDEA analyzes the Maven POM and offers stuff you define as depenendcy in there
14:48 pdurbin Well, Netbeans is definitely operating off the POM. But I don't know how it decides what to suggest.
14:49 pdurbin poikilotherm: your commit above makes sense. Does it break anything? Or change how we deploy the war file? Do you want to go ahead and make a pull request?
14:50 poikilotherm Do you think I should split up? We talked about that a few days ago.
14:50 poikilotherm Splitting is not difficult, I'm just unsure.
14:51 poikilotherm What do you prefer?
14:51 poikilotherm I hadn't a chance to try agains some S3 yet
14:51 pdurbin smaller chunks are almost always better
14:51 poikilotherm Maybe you guys can give this a shot?
14:52 pdurbin It's great when developers let QA know if they have concerns about any particular functionality, such as S3 or anything else.
14:53 poikilotherm Should I add some docs about the maven deps management first?
14:54 poikilotherm Maybe it makes sense to create a successor to my AWS issue describing a more general dependency cleanup
14:54 poikilotherm Actually I made the first steps towars this...
14:54 pdurbin Yes, a new issue might be best. And yes to docs. Always docs, if you're willing to write them!
14:55 poikilotherm Sure. Any suggestions about a place within the dev docs? Browsed through them, but unsure where this might fit best
14:56 pdurbin Hmm. http://guides.dataverse.org/en/4.9.4/developers/tips.html maybe?
14:56 pdurbin The tips page is kind of a dumping ground. :/
14:56 pdurbin http://guides.dataverse.org/en/4.9.4/developers/coding-style.html maybe? A new section about Maven?
14:57 pdurbin Or a new page?
14:57 pdurbin A new page called "dependency management"?
14:58 poikilotherm Let's make a new page... Tips and Code Style seem to cover other topics, but this is definitly something noteworthy to any developer
14:58 poikilotherm Thus I think it deserves its own entry in the TOC on the left
14:59 pdurbin Sure, please go for it.
15:00 pdurbin Eventually we might need some nesting in that sidebar.
15:01 pdurbin MrK: at standup in about an hour I'm planning on mentioning how you'd like to contribute. Thanks again for the offer!
15:09 MrK pdurbin: oh thank you! Would it be possible to mention an access to slack so there would be an possibilty to talk with the design team? Btw. There are 2 developers from us assigned to this project at the moment.
15:20 poikilotherm pdurbin: https://github.com/IQSS/dataverse/issues/5288
15:23 pdurbin poikilotherm: cool. Thanks! I don't know what a BOM is.
15:24 pdurbin oh, Bill Of Materials
15:25 poikilotherm And to be precise: https://github.com/IQSS/dataverse/pull/5289
15:26 pdurbin poikilotherm: thanks! Dragged to the "Community Dev" column at https://waffle.io/IQSS/dataverse
15:26 poikilotherm :-)
15:27 poikilotherm Maybe you can assign matthew, scolapasta and me?
15:27 poikilotherm More workload on people in charge... :-P
15:27 pdurbin They aren't in the office yet.
15:27 poikilotherm Oh! Alright...
15:27 pdurbin I'll mention it at standup in 45 minutes.
15:28 poikilotherm Thank you very much! :-)
15:28 pdurbin sure, I have a list of three things to bring up
15:28 pdurbin If anyone has anything else, please let me know. :)
15:28 poikilotherm Lets see if I can add more to that... :-D
15:28 poikilotherm What's already on the list?
15:29 pdurbin 1. MrK and another developer want to contribute features with UI/UX impact.
15:29 pdurbin 2. AccessIT is failing: https://github.com/IQSS/dataverse/issues/5287
15:30 pdurbin 3. Oliver wants to talk to Gustavo and Matthew about upgrading Glassfish, deps, etc.
15:30 poikilotherm About Nr. 2: isnt that one of the tests that fail randomly?
15:31 pdurbin No, I'm hoping it's just the Solr schema change I added.
15:31 poikilotherm OK
15:31 * poikilotherm crosses fingers
15:32 pdurbin :)
15:32 poikilotherm Oh about the Community Dev label: maybe that was a bit to quick? I dunno if matthew and scolapasta might prefer to do this on their own, as it is kinda reaping out body parts replacing it with fresh stuff
15:33 MrK joined #dataverse
15:33 poikilotherm I'm not sure if the trust an outside community dev enough for this
15:33 pameyer joined #dataverse
15:34 poikilotherm This is no prio for me, my primary goal is still #5274
15:39 pdurbin Oh, I thought 5288 was something you wanted.
15:40 poikilotherm Sure - I will need it as a base for 5274
15:41 poikilotherm But if there is no interest in the rest of 5287, I'm happy to leave it "as is"
15:41 poikilotherm By the way: the WAR is at 125MB now... ;-) -21MB, yeah!
15:42 donsizemore joined #dataverse
15:45 pdurbin I don't think you mean 5287.
15:45 poikilotherm Argh.
15:45 poikilotherm You are right
15:46 poikilotherm The rest of 5288
15:46 pameyer pdurbin: I'd thought there was another one of the ITs failing too; but from a quick look don't see the issue. any ideas if that one was fixed?
15:48 pdurbin pameyer: you might be thinking of https://github.com/IQSS/dataverse/issues/5260
15:49 pdurbin poikilotherm: yeah, I'm not sure. I trust you three will decide what's important to you. I'm interested in the ability to move off of the old version of Glassfish we're stuck on.
15:49 pameyer pdurbin: yes; thanks
15:52 pdurbin poikilotherm pameyer: for the record, the AccessIT thing was just the Solr schema. Thanks for crossing fingers. Sorry for the noise.
15:54 pameyer nice to see I'm not that sometimes opens issues where the resolution is "requestor got mixed up" :)
15:56 pdurbin I forgot I even messed with the Solr schema.
15:56 pdurbin But I just emailed dataverse-dev to spread the word.
15:56 pameyer you managed to include it in docker-aio, so it passed there
16:03 pdurbin sure, docker-aio is smarter about setting up solr automatically than we developers are on our laptops :)
16:06 donsizemore joined #dataverse
16:25 poikilotherm Somebody asked about security scans at community call... Maybe https://projects.eclipse.org/proposals/eclipse-cognicrypt can help out with scanning for crypto security?
16:42 isullivan joined #dataverse
16:50 pdurbin poikilotherm: maybe, pameyer has some tools he uses too
16:50 pdurbin poikilotherm: I let those guys know you want to talk app servers and deps
16:55 pameyer pdurbin poikilotherm there are some security scanning tools I use; but with my paranoia level I'm not a bit fan of going into too much detail on those in a world-visible publicly logged forum
16:56 pdurbin pameyer: excellent point. poikilotherm and others, if you find vulnerabilities, please report them to security@dataverse.org and you will get a support ticket number for tracking purposes. That address is mentioned in CONTRIBUTING.md.
16:57 poikilotherm Yes Sir! :-)
16:58 poikilotherm Thx for mentioning at stand up :-)
16:59 poikilotherm I'm back on work on 4172 and getting the docker based stuff inside the pom. It is such a blessing not to worry about the hacky stuff anymore - that gave me some headaches...
17:01 pdurbin Cool, for what it's worth nobody seemed to object to Payara at tech hours yesterday.
17:04 poikilotherm Glad to hear that :-)
17:05 pdurbin :)
17:13 nanoz joined #dataverse
17:29 poikilotherm pdurbin: IIRC you where the one in charge to write the install scripts, right?
17:30 poikilotherm Why has the postgres JAR dependency been managed outside of the POM?
17:30 poikilotherm Wouldn't it make sense to include that?
17:30 poikilotherm Obviously there is some historic reason to that...
17:40 jri joined #dataverse
17:41 pameyer there used to be several postgres JARs bundleded with the installer, and which one was used with glassfish depended on the postgresql version
17:42 pameyer then the postgres jdbc driver supported all the postgresl server/client versions dataverse was targeting, and there wasn't a need for multiple jars
17:43 poikilotherm Alright - and why has this not moved inside the WAR?
17:43 poikilotherm Wouldn't that be easier for maintenance?
17:44 poikilotherm (Sometimes new versions of the jdbc JAR are released for various reasons. That will never be updated or only when the install script is re-run on an update of dataverse)
17:44 poikilotherm Packaging it inside the WAR might be easier then, isn't it?
17:51 poikilotherm Alright guys, that's it for me for today... Cu all tomorrow :-)
17:51 poikilotherm Good night / Good afternoon ;-)
18:27 donsizemore joined #dataverse
18:30 pdurbin It wasn't me! :)
18:48 pdurbin ... though I did add "use strict" :)
19:29 donsizemore joined #dataverse
20:12 jri joined #dataverse
20:29 isullivan joined #dataverse
20:48 JonathanNeal joined #dataverse
21:14 pameyer @poikilotherm something that only occured to me later - the install script copies the postgres jar before war deployment; and some of the glassfish configuration to create the database connection pool happenes before war deployment.
21:16 pameyer I don't have a lot of expertise w\ glassfish / j2ee connection pools, but I think that if the postgres was in the war, creating the connection pool would fail.  and I wouldn't be surprised if deploying the applicaiton required the connection pool to exist (because if not, that configs could have been done elsewhere)
23:23 poiki_at_home joined #dataverse
23:24 poiki_at_home pameyer: just read your comment about the connection pools. Maybe this was true for Glassfish 3 and 4, but at least with 5 this gets a lot easier: https://blog.payara.fish/an-intro-to-connection-pools-in-payara-server-5#id-[MARK-434]AnIntroductiontoConnectio​nPoolsinPayaraServer-CreatingtheCon​nectionPool-Manuallyviadomain.xml
23:24 * pdurbin hopes poiki_at_home gets enough sleep tonight ;)
23:25 pameyer @poiki_at_home good to hear
23:25 pameyer and yeah, downtime is good :)
23:25 poiki_at_home Ah damn, took the wrong section
23:25 poiki_at_home https://blog.payara.fish/an-intro-to-connection-pools-in-payara-server-5#id-[MARK-434]AnIntroductiontoConnectio​nPoolsinPayaraServer-CreatingtheCon​nectionPool-PerApplicationResources
23:26 poiki_at_home I really would like to see less asadmin commands when fiddling around with containers...
23:26 poiki_at_home Makes the building process much easier ;-) Just deploy and go go go
23:27 pdurbin What's the alternative? XPath madness?
23:27 pameyer less asadmin would definately be better
23:28 pameyer it would need code changes, but moving stuff to environmental variables still seems like a not-horrible idea (at least, for where it can be done)
23:29 pdurbin I have no quarrel with environment variables. :)
23:30 pameyer my turn to head out for the night ;)
23:31 poiki_at_home Nope, the alternative is configure the JDBC stuff from inside the application
23:32 pdurbin Sounds fancy.
23:33 poiki_at_home http://dplatz.de/blog/2018/self-contained-jee-app.html
23:33 poiki_at_home That's a pretty good summary of how to get there ... :-)
23:33 poiki_at_home Payara and Wildfly lets you reuse ENV vars within annotations
23:33 poiki_at_home Thats really cool stuff
23:34 poiki_at_home Maybe this is an option for those other config parts, too? (Need to dig deeper into that...)
23:34 poiki_at_home No more fiddling with asadmin... Ah, what a dream
23:35 poiki_at_home Mainly because of asadmin not being idempotent
23:36 poiki_at_home Oh and OmniFaces did some fancy stuff, too ;-) http://jdevelopment.nl/switching-data-sources-datasourcedefinition/
23:36 pdurbin As long as we don't go back to a bunch of replaces with sed on an XML file, I'm fine with whatever.
23:38 pdurbin Those were dark days.
23:39 poiki_at_home NO WAY!
23:40 poiki_at_home I will stand by your side, fighting everyone trying to get over that wall
23:40 poiki_at_home That area has been sealed... ;-)
23:41 poiki_at_home Alright - 00:40 over here. Finally heading for my bed now, more stuff tomorrow.
23:41 poiki_at_home Cu all
23:41 * pdurbin hands poiki_at_home a halberd
23:42 poiki_at_home left #dataverse
23:48 pdurbin huh, this dplatz solution does look pretty good

| Channels | #dataverse index | Today | | Search | Google Search | Plain-Text | plain, newest first | summary

Connect via chat.dataverse.org to discuss Dataverse (dataverse.org, an open source web application for sharing, citing, analyzing, and preserving research data) with users and developers.