Time
S
Nick
Message
00:41 jri joined #dataverse
02:57 jri joined #dataverse
05:13 jri joined #dataverse
07:41 jri joined #dataverse
08:07 jri joined #dataverse
08:46 MrK joined #dataverse
08:51 poikilotherm joined #dataverse
09:24 MrK
Hi poikilotherm, regarding https://github.com/IQSS/dataverse/pull/5349#discussion_r238247379 you by value you mean like a field?
09:36 poikilotherm
Good morning :-)
09:36 * poikilotherm
reads the comment again
09:37 poikilotherm
Yeah. To be more in OOP speech: a class attribute
09:37 poikilotherm
Edited my comment to be more precise
09:39 poikilotherm
Also it is unlikely to happen: data migrations should be described or at least mentioned, too
09:39 poikilotherm
Those are dangerous and might use lots of resources
09:50 MrK
But adding the field woudn't be the same since we have an option to only create tables not add fields, so if someone already has the table adding the field would not add the column to the table.
09:54 poikilotherm
Right
09:54 poikilotherm
But normally when you add a field this should be persisted, isn't it
09:54 poikilotherm
Of course there are cases where this is not true
09:55 poikilotherm
And shouldn't those changes have a migration, too? Otherwise this would lead to KABOOM, wouldn't it?
09:58 MrK
poikilotherm: That depends on persistence.xml setting, but we only have an "create-tables" option https://www.eclipse.org/eclipselink/documentation/2.5/jpa/extensions/p_ddl_generation.htm so new fields wont be persisted to my understanding
09:59 MrK
Yeah new field's should be added in the entity class and in the migration sql
10:09 poikilotherm
I wonder how this has been handled to date
10:09 poikilotherm
Maybe pdurbin can enlighten us
10:09 poikilotherm
Most certainly he will be with us in about 2h ;-)
10:11 poikilotherm
He or his kids seems to like worms, as he really is an early bird ;-)
10:43 MrK
Well I can tell you how it was handled :P, every change was added to upgrade script and after new release you had to execute this sql .
10:49 MrK
I wish I could sleep like 6 hours and have enough of sleep, I sleep over 7,5 and I still feel like i need more.
10:52 MrK
poikilotherm: I'm wondering if I should start working on this previous sql upgrade migrator or wait for the landreev decision
11:03 poikilotherm
Maybe we should try to follow documentation driven like pdurbin said yesterday
11:04 poikilotherm
You could stuff about that in a developer doc and add it to your PR
11:11 pdurbin joined #dataverse
11:11 pdurbin
no worms today, the ground is frozen
11:15 poikilotherm
Good morning pdurbin :-)
11:15 pdurbin
mornin'
11:16 poikilotherm
MrK: I was wrong. It has been just 1h.
11:16 poikilotherm
pdurbin: Man, when do you go to bed? 6pm? I would be DEAD getting up that early and already texting to the other side of the world.
11:17 poikilotherm
Thank you for ongoing contribution in this community channel. :-)
11:17 poikilotherm
I really appreciate that and cannot emphasize enough your effort in getting us together
11:18 pdurbin
I think I slept 8 hours. 10pm until 6am or so. I'm fine. But have you seen those t-shirts that say, "Ok, but first coffee"? :)
11:19 poikilotherm
LOL
11:20 pdurbin
https://github.com/IQSS/dataverse/blob/092ea01af583670aebbf66fae659c293ca6a037d/doc/sphinx-guides/source/developers/sql-upgrade-scripts.rst
11:20 poikilotherm
Go grab it ;-) We will wait stay patient till you're back
11:20 pdurbin
^^ that's what I wrote about SQL script (pre flyway). Is there a question about our current process?
11:21 poikilotherm
Err - this is commited on 10/16/2018
11:21 poikilotherm
How has this been done before?
11:21 poikilotherm
You know back in the middle ages of DV 4.0
11:37 pdurbin
The same way. I finally wrote it down.
11:37 pdurbin
We've always done things this way.
11:46 poikilotherm
Ok, good to know
11:46 * poikilotherm
is digging through timer code
12:07 pdurbin
poikilotherm: oh, I'm slightly less excited about Gitter because the other day one of the developers said, "We are focused on the SaaS experience, gitter.im. Self installations is not a route we plan to support but people are free to run their own."
12:08 pdurbin
I mean, I'm fine with using SaaS products. I use GitHub, Travis, Coveralls, etc. But I like it a bit better when it's relatively each to install it yourself, like Travis or GitLab.
12:17 pdurbin
MrK: I did talk to Leonid after standup about Flyway. He seems to like the idea. He wants to play with it himself, which is something I haven't done yet.
12:17 MrK
pdurbin: Ok cool, so I understand I should just wait for comments in PR?
12:18 pdurbin
Yeah, I think you should just wait. This is all new to us.
12:18 pdurbin
I think you've been following the original issue... we want upgrades to be easier.
12:19 MrK
Yeah flyway should solve database schema upgrade issue
12:20 pdurbin
So our thinking is that people on older version of Dataverse (e.g. 4.6.2) will use Leonid's new script to upgrade to a version of Dataverse with Flyway (maybe it will be merged in Dataverse 4.10?). And then, migrations will be handled by Flyway instead of Leonid's script.
12:20 pdurbin
I hope that makes sense. That's our understanding anyway.
12:31 MrK
Well that depends pdurbin right now you would have to upgrade to the version before flyway, and then to the flyway version. If I implement poikilotherm idea you wouldn't even have to do that you would just download the new version.
12:38 pdurbin
MrK: I'm fine with whatever. I just want the steps for people upgrading to be clear.
12:40 pdurbin
MrK: It's hard for us to keep up with all the ideas you and poikilotherm have. Looking at pull requests is easier. :)
12:40 poikilotherm
LOL
12:40 pdurbin
You guys are from the future. We're still banging rocks together.
12:40 poikilotherm
pdurbin I am editing docs for timers to follow your suggestion about Doc driven dev
12:42 pdurbin
poikilotherm: sounds good. I think Gustavo calls this "pre review" to distinguish it from "code review". I'm the one who likes the term "documentation driven development." He would probably just have a conversation about the idea before coding begins.
12:42 poikilotherm
Let's call it D-D-D. Sounds impressive ;-)
12:43 pdurbin
scales better too
12:43 poikilotherm
"Hey man, our software is built using D-D-D!"
12:44 poikilotherm
That will make an impression on people, won't it? ;-)
12:44 pdurbin
"Really? 20 sided dice? Swords and magic wands?"
12:46 pdurbin
"No, not D&D. DDD."
12:46 poikilotherm
ROFL
14:16 MrK
DDD is already taken :D
14:18 poikilotherm
Yeah - indicates all digital workflow on audio cds ;-)
14:20 poikilotherm
And of course by domain driven design
14:25 MrK
Yeah, do you have the knowledge about domain driven design? I've read that people even after reading still don't know what is it exactly :D, and people who think they know they are doing it completely wrong :D.
14:28 donsizemore joined #dataverse
14:28 donsizemore
hey @pdurbin just a heads up we're about to make the jump to 4.9.4
14:30 pdurbin
donsizemore: punch it, Chewie!
14:32 poikilotherm
MrK: I just had a read or two about this on heise dev, but not more...
15:39 poikilotherm
pdurbin: https://github.com/IQSS/dataverse/issues/5345#issuecomment-444529858
15:41 poikilotherm
Gustavo is not all happy with your suggestion, but lets see where this takes us. He mentioned that IQSS still has to figure out how to deal with community dev efforts... ;-)
16:07 pameyer joined #dataverse
17:53 jri joined #dataverse
18:28 pameyer
pdurbin: do you know if Sherry got things sorted out with her stability issues?
18:30 pdurbin
pameyer: good question, I've been meaning to look for an RT ticket
18:32 pdurbin
pameyer: I don't see a ticket so if you see her in here, please ask her for the number. Maybe I simply can't find it.
18:45 pameyer
pdurbin: thanks for taking a look
19:12 jri joined #dataverse
19:19 pdurbin
sure
19:20 pdurbin
xarthisius: I'm happy to see all the Dataverse-related pull requests in Whole Tale repos!
19:20 xarthisius
We're missing docs, json manifest, and we owe you a PR to external tools listing :)
19:21 xarthisius
we aim to release it really soon
19:33 jri joined #dataverse
19:54 jri joined #dataverse
20:15 jri joined #dataverse
20:20 donsizemore joined #dataverse
20:21 donsizemore
@pdurbin knock knock?
20:25 dataverse-user joined #dataverse
20:26 pdurbin
xarthisius: I was wondering if the manifest is in your docs yet. Is there a pull request yet? :)
20:26 pdurbin
donsizemore: hi! How'd the upgrade go?
20:26 donsizemore
@pdurbin i think it's monday. dumb question?
20:26 pdurbin
hit me
20:27 donsizemore
EZID reserved DOIs and DataCite. We see that Dataverse checks for potential DOI conflicts and will mint a new one if it thinks there's a problem
20:27 donsizemore
but we're curious why you all made your reserved DOIs public during your migration
20:27 donsizemore
this should probably go to Leonid, but... you're here =)
20:30 pdurbin
Buh. I don't know. I didn't know we made reserved DOIs public.
20:30 pdurbin
But I'll take your word for it.
20:32 pameyer
@donsizemore - I don't really know, but I have vague ideas that it was a ezid -> datacite thing, rather than a dataverse thing
20:32 pameyer
but it's very possible I'm wrong
20:33 donsizemore
@pdurbin and, following the Principle of Maximum Embarrassment, I now see that only _file_ DOIs were made public
20:33 donsizemore
@pdurbin we were looking at datasets (thu-mai doesn't want to do file-level registration yet)
20:35 jri joined #dataverse
20:53 poikilotherm joined #dataverse
20:56 jri joined #dataverse
20:58 pdurbin
donsizemore: the file level DOIs reminds me of an issue Jim worked on. One sec.
21:00 pdurbin
donsizemore: this one: https://github.com/IQSS/dataverse/issues/4778
21:00 pdurbin
I'm not sure if that helps or not.
21:00 pdurbin
dataverse-user: hi! Welcome!
21:01 donsizemore
@pdurbin oooh, i forgot about that one. thanks!
21:01 pdurbin
sure
21:13 poikilotherm
Hi pdurbin - please stop me if you think this is not the right place to discuss this. Gustavo mentioned that in the past you guys successfully had some interns (and I know about Redhat, ...) and that he would be open for people like me to come over and work with you for a week or two ("we have the space"). Do you think there could be options in terms of finance for such a thing? I am pretty unsure if FZJ
21:13 poikilotherm
will fund that, or if my bosses like the idea, but it seems worth a question
21:17 jri joined #dataverse
21:18 pdurbin
poikilotherm: I doubt IQSS would pay for it but who knows.
21:19 poikilotherm
Maybe there could be some kind of split?
21:20 pdurbin
I like your optimism.
21:20 poikilotherm
Are there options at Harvard for cheap rooms for such a stay? A hotel is most likely to expensive
21:20 poikilotherm
+
21:20 poikilotherm
+o
21:20 pdurbin
The person to ask is probably Michael Bar-Sinai.
21:21 pdurbin
https://github.com/IQSS/dataverse/commits?author=michbarsinai
21:22 pdurbin
for rooms near Harvard I mean
21:22 pdurbin
maybe you've seen his name in the code :)
21:23 pdurbin
Hey, it says " I'm also michbarsinai on freenode, I normally hang out at #dataverse." at http://mbarsinai.com ... maybe he means when he's signed in, he's here. Not that he's here all the time. :)
21:24 poikilotherm
:-D
21:24 pdurbin
poikilotherm: anyway, it would be great if you could come hang out for a while. You should ask your boss to fund it. :)
21:24 pdurbin
Or you could fly me out there. :)
21:25 poikilotherm
Household government funding is the more difficult part... ;-)
21:25 poikilotherm
She needs to look after the kids for some days, which is ... well ... exhausting
21:26 pdurbin
don't I know it
21:26 pdurbin
I was allowed to go to Indonesia. Nice trip. Couldn't stay too long. Because kids.
21:35 poikilotherm
Alright I will ask the people in charge if they like the idea in the first place
21:36 poikilotherm
Someone needs to fund the flights at least... ;-) And I heard from american friends those aren't cheap...
21:37 pdurbin
Probably not.
21:37 jri joined #dataverse
21:38 pdurbin
Sometimes people come for the community meeting and stick around for a few days after. It's in June.
21:38 pdurbin
The weather is better in June. :)
21:39 poikilotherm
LOL
21:39 poikilotherm
But June is pretty far away
21:40 poikilotherm
I was thinking if the recent stuff would be easier to get done when I am with you guys
21:40 pdurbin
oh absolutely
21:40 pdurbin
I mean, it sure seems like you're spending a fair amount of time thinking about Dataverse. It would be a shame to squander all that effort.
21:41 pdurbin
I want to keep you unblocked.
21:41 pdurbin
Are you blocked now?
21:41 pdurbin
Leonid said he'd look at your EJB timer thing.
21:41 pdurbin
and he's looking at the Flyway stuff.
21:42 pdurbin
which you seem to want
21:42 poikilotherm
That sounds promising :-D
21:42 xarthisius
pdurbin: one issue with manifest file is the location of WT. In principle, there might be other instance people'd like to use. Any hints how to deal with that?
21:43 xarthisius
*instances
21:43 poikilotherm
Glad you guys give us some higher prio on your list of stuff to look at
21:44 pdurbin
poikilotherm: oh, another thought is that during the community call I'm not sure how many people in the room or on the call had heard of Kubernetes. I think xarthisius was called in, so maybe more that I think. :)
21:44 poikilotherm
LOL
21:44 poikilotherm
Maybe I should use more buzzwords?
21:45 poikilotherm
"cloud native" to start with
21:45 poikilotherm
(But honestly I don't like those...)
21:45 xarthisius
yeah, what's a "container" ? :)
21:45 pdurbin
xarthisius: I think Whole Tale suffers from the same thing Dataverse has suffered from for years, which we've recently started calling "flagship bias". You use "Whole Tale" to mean your software, the installation you run, everything. In Dataverse I have been pushing to distinguish more between the software and the installation Harvard runs (1 of 35).
21:47 poikilotherm
xarthisius: that was rhetoric, wasn't it?
21:47 xarthisius
maybe I'm worring preemptively, there's gonna be just one WT and it will wither and die when the grant will run out :)
21:47 pdurbin
poikilotherm: maybe we should have a follow up call for the people who were excited about the discussion. A lot of people in the room didn't know what you were talking about. It's not their job to. They do design or metadata, etc.
21:48 poikilotherm
:-)
21:48 pdurbin
xarthisius: sure, that's always a concern. But we keep referring to Whole Tale as an open source Code Ocean that institutions can self host.
21:48 poikilotherm
People that normally just benefit from things like DevOps, but don't do that themselfs... ;-)
21:49 poikilotherm
About the follow up: good idea
21:49 pdurbin
Sure. People benefit from flipping on a light switch too. They don't need to know the details. :)
21:49 poikilotherm
Do you know whom to ask? I don't have a good oversight about who attended the call and was interested...
21:49 poikilotherm
+
21:50 pdurbin
poikilotherm: actually... I think I started a spreadsheet about people who are interested in Docker and Kubernetes. One sec.
21:51 xarthisius
I made a note https://github.com/whole-tale/wt-design-docs/issues/61#issuecomment-444663468
21:52 pdurbin
xarthisius: looks good. You could also look at how Data Explorer explains how to use its manifest. One sec.
21:53 xarthisius
looks like they have 2 files at least
21:53 xarthisius
one for local deployment, and one for remote
21:54 pdurbin
ah, yes, good catch
21:55 pdurbin
I mean, do you want people to use your installation?
21:55 xarthisius
yup
21:55 xarthisius
on wholetale.readthedocs.io we will host json pointing to our "production" deployment
21:55 pdurbin
Ok, so maybe just leave a note that it can also be used on one's own installation of Whole Tale and link to your installation guide.
21:56 xarthisius
are you validating somehow whether `toolUrl` actually exists and responds ?
21:56 pdurbin
poikilotherm: I found the conversation in a thread called "even more Kubernetes interest" but I don't think I ever made a spreadsheet. Sorry.
21:57 pdurbin
xarthisius: nope.
21:58 poikilotherm
Do you think a channels like twitter plus might reach people to head for a poll about finding a fellow up call time slot?
21:59 poikilotherm
-e+o
21:59 poikilotherm
Oh dear, its late
21:59 pdurbin
poikilotherm: Twitter might work but I'd suggest creating a Doodle poll or similar, sending it to the dataverse-community mailing list
21:59 poikilotherm
Do you think a message via channels like twitter plus email might reach people to head for a poll about finding a follow up call time slot?
22:00 pdurbin
From your tweet you could link to your post on the mailing list.
22:00 poikilotherm
Yeah
22:01 poikilotherm
I don't know if we can reach those chinese guys the australian guy mentioned (sorry, lost his name... :-( )
22:02 pdurbin
I only know the, uh, higher up people. Not the sysadmins.
22:02 pdurbin
For Fudan and ADA.
22:03 pdurbin
really, you want to reach the sysadmins
22:03 poikilotherm
Then lets hope follower power will just work :-D
22:03 pdurbin
these type of people can sometimes be found on IRC :)
22:03 poikilotherm
Yeah, definitly
22:03 poikilotherm
*G*
22:04 poikilotherm
I think dataverse is actually the first thing in about 15 years that let me get in touch with IRC again
22:04 pdurbin
Well, we like standards. RFCs.
22:12 poikilotherm
https://doodle.com/poll/tnbferwypuib26wx
22:13 pdurbin
ok, so you're thinking next week. sounds fine
22:14 pdurbin
poikilotherm: and you get to pick the technology we use to call in :)
22:14 pdurbin
poikilotherm: have you seen how Craig uses GitHub issues to organize the calls that xarthisius and I call into?
22:15 poikilotherm
Most certainly I will just use conf.dfn.de or try Zoom. Zoom allows 40 minutes for free, that should be ok for a follow up call
22:15 poikilotherm
Nope
22:16 pdurbin
Please see https://github.com/whole-tale/whole-tale/issues/43 and https://github.com/whole-tale/whole-tale/issues/50
22:16 pdurbin
poikilotherm: you're welcome to create a Dataverse issue for this if you want. Up to you.
22:17 pameyer
some communities are more into twitter than others
22:17 pdurbin
pameyer: truer words were never said
22:18 poikilotherm
Oh this is a pretty good idea, I like it
22:19 poikilotherm
Will try to get some feedback in the poll first
22:19 poikilotherm
Hmm
22:19 poikilotherm
On the other hand it might be easier to point people to such an issue and ask them to leave their feedback there
22:20 poikilotherm
E.g. mention people that they think might be interested
22:20 pdurbin
Yeah, and Craig always creates a Google doc for agenda/notes.
22:20 pdurbin
It's a pretty good pattern.
22:21 pdurbin
I like being able to help with the agenda in a Google doc rather than spamming people with little comments in a GitHub issue.
22:22 pdurbin
poikilotherm: I have a "GitHub Issues By Username" spreadsheet. I'm thinking I'll add a "Kubernetes" column.
22:24 pdurbin
There are 141 GitHub usernames in this thing. I'm trying to remember who is who. :)
22:30 poikilotherm
https://github.com/IQSS/dataverse/issues/5373
22:31 pdurbin
poikilotherm: nice, want me to create a Google doc for an agenda/notes under our "community" folder?
22:33 poikilotherm
https://twitter.com/poi_ki_lo_therm/status/1070446102434979842
22:34 poikilotherm
Maybe you guys could retweet that? YOu have more followers... :-D
22:35 poikilotherm
Yeah, Google Doc sounds good :-)
22:36 pdurbin
poikilotherm: retweeted. From my personal account. I don't control @dataverseorg. Here's a doc: https://docs.google.com/document/d/13h4zra_ugYH5IKMkj5tya1NwaFdC2FfWXBmGUxxAdn8/edit?usp=sharing
22:36 poikilotherm
pdurbin: Thx!! :-)
22:45 pameyer
poikilotherm - I don't remember the usernames, but I don't think the openshift/redhat folks got mentioned in 5373
22:46 pameyer
not sure where it is on their list of priorities
22:49 poikilotherm
Yeah... Maybe I look up the usernames tomorrow from the git commits
22:50 poikilotherm
Right now it is 23:50 over here... Bedding time again
22:50 poikilotherm
Have a good evening guys
22:50 poikilotherm
(Maybe I should bring a can of worms tomorrow? ;-) )
22:51 pdurbin
poikilotherm: I have some usernames for you
22:52 poikilotherm
Great! Shall I edit my comment or do you want to comment yourself?
22:52 pdurbin
I'll leave a comment. Probably tomorrow.
22:52 poikilotherm
Thx! :-)
22:52 poikilotherm
No worries
22:53 pdurbin
poikilotherm: and you should email dataverse-community
22:53 poikilotherm
Yeah :-)
22:53 poikilotherm
Will do that tomorrow
22:53 * poikilotherm
hits the keyboard with his forehead
22:53 poikilotherm
Good night... Need some sleep now :-)
