Time
S
Nick
Message
07:15 juancorr joined #dataverse
08:37 stefankasberger joined #dataverse
09:05 poikilotherm joined #dataverse
09:55 pdurbin joined #dataverse
09:55 pdurbin
I hope everyone had a good weekend.
10:02 poikilotherm
Good morning America :-)
10:03 poikilotherm
Yeah, was fabulous
10:03 pdurbin
glad to hear it
10:04 poikilotherm
The last two days of summer I guess
10:05 poikilotherm
Kids were happy and I was happy while taping foil in my roof ;-)
10:05 poikilotherm
pdurbin I think this episode of airhacks might be interesting for you... https://twitter.com/poi_ki_lo_therm/status/1176052408847413248
10:06 pdurbin
#53 From PHP to Transactions? Yeah, it was pretty good.
10:06 poikilotherm
No, #54 or #55
10:07 poikilotherm
(Maybe #56, dunno how much in advance Adam is planning the topics)
10:07 pdurbin
Oh, a future episode. I see.
10:08 pdurbin
While we wait, a few days ago the recordings from JakaraOne went up on YouTube. They are linked from https://jakartaone.org
10:08 poikilotherm
I recently watched some presentations by Kevin on testcontainers
10:08 poikilotherm
Cool
10:08 poikilotherm
Speaking of Jakarte
10:08 poikilotherm
-e+a
10:09 poikilotherm
Should we take the opportunity and switch Dataverse to Jakarta in our branch?
10:09 pdurbin
Sure, why not.
10:10 poikilotherm
Oh Slava just send me an email
10:10 poikilotherm
https://github.com/ekoi/dataverse-kubernetes/blob/ddi-converter-tool/README-AWS-Dataverse.md
10:10 poikilotherm
They are actively working on that
10:10 pdurbin
nice
10:10 poikilotherm
He is asking me if I would like to see it in dataverse-kubernetes :-D
10:11 poikilotherm
Do you think it might be a good idea to start a guide with the look and feel of Dataverse guides?
10:12 pdurbin
I'm fine with whatever. As you know, http://guides.dataverse.org/en/4.16/developers/containers.html need a rewrite. But we can worry about that later.
10:13 poikilotherm
I was thinking about refactoring the docs at dataverse-kubernetes
10:13 poikilotherm
And use sphinx for those
10:13 poikilotherm
And ask for permission to use the Dataverse skin templates
10:14 pdurbin
You have my permission.
10:14 pdurbin
Someday maybe we can have an additional guide.
10:14 poikilotherm
Yeah, that's what I was up to
10:15 poikilotherm
All this container stuff is moving fast
10:15 pdurbin
Because right now the k8s and Docker stuff is wedged into the dev guide. But it's not just a dev thing.
10:15 poikilotherm
Aye
10:15 poikilotherm
Maybe it should live somewhere else
10:15 pdurbin
It's almost like you're trying to answer the original question at https://github.com/IQSS/dataverse/issues/4665 which is "Is there official production ready docker container for data verse ? and if available where can i find the docs?"
10:16 poikilotherm
Yeah, why not?
10:16 poikilotherm
If it matures to that, I'll be ok with that
10:17 juancorr67 joined #dataverse
10:20 pdurbin
We should feel like we need to refactor that "containers" page much. We can just delete it. I can add back in anything I care about.
10:21 poikilotherm
We could migrate usefull stuff into the new guides
10:22 poikilotherm
guide
10:22 poikilotherm
Not multiple guides
10:22 poikilotherm
https://github.com/IQSS/dataverse-kubernetes/issues/92
10:22 pdurbin
looks good
10:29 poikilotherm
Thx
10:47 pdurbin
I'm hacking on the new "meetups" spreadsheet. Just added an "About" tab. Would anyone like to take a look? Does anyone have any meetups to add? https://docs.google.com/spreadsheets/d/1EQsrT0A-zBQxN9L8iJBW-7auECWvFW2qu8r-CbWrTEw/edit?usp=sharing
10:49 poikilotherm
Did Danny say anything about Hacktoberfest?
10:50 poikilotherm
You could mention DCM 2020 ;-)
10:51 poikilotherm
And it would be pretty awesome to have a column mentioning who from IQSS will be there
10:52 poikilotherm
That might have some influence on attending to some event due to meeting someone special
10:56 pdurbin
I don't know the dates for DCM 2020 yet.
10:57 poikilotherm
Ok, you could list it anyway, couldn't you?
10:58 poikilotherm
And maybe sth. generic as "Summer 2020. tba"
10:58 poikilotherm
So people remember and keep it in mind ;-)
10:58 pdurbin
Hmm, but I'm using a date format.
10:58 pdurbin
I figure we'll want to keep it sorted.
10:58 poikilotherm
Hmm
10:59 pdurbin
I also just added something to https://dataverse.org/events ... it's a little long but I hope it looks ok. At the bottom.
10:59 poikilotherm
You could use start and end date
10:59 pdurbin
Less is more so I'm just trying start date for now. :)
10:59 poikilotherm
Ok :-)
11:00 pdurbin
Think microprofile not java ee.
11:00 poikilotherm
Which kind of lambda expression do you prefer in Java?
11:00 poikilotherm
Declarative or function reference?
11:00 pdurbin
I don't know the difference.
11:00 poikilotherm
https://www.logicbig.com/how-to/java/lambda-list-contains-a-substring.html
11:00 pdurbin
Whatever Netbeans autocompletes it to. :)
11:01 pdurbin
I've seen both of these forms. No real preference.
11:02 pdurbin
We could add something to a future version of http://guides.dataverse.org/en/4.16/developers/coding-style.html#java if you want.
11:02 pdurbin
What about past events?
11:03 poikilotherm
Heh, you could create a Twitter timeline :_D
11:20 pdurbin
less is more :)
11:20 pdurbin
We already have https://dataverse.org/presentations which is sort of a list of past events.
11:51 poikilotherm
pdurbin I catched my exception with the scope...
11:51 poikilotherm
https://pastebin.com/deG810pt
11:52 poikilotherm
Untested code is... Interesting.
11:53 poikilotherm
I have to leave for an appointment now. Read you later :-) I'm anxious if you see the missing piece ;-)
12:17 pdurbin
looks like scope is null
12:37 donsizemore joined #dataverse
12:47 dataverse-user joined #dataverse
13:37 pdurbin
donsizemore: good morning! I see you pushed a commit or two!
13:37 donsizemore
@pdurbin good morning — yes but i'm not done yet =(
13:38 donsizemore
i just fixed a dataverse issue here, need to fire off a note to jon and i'll be back on that.
13:38 pdurbin
ah, ok, I'll wait then... need any help?
13:38 donsizemore
while you're here, though, google dataset search question?
13:38 pdurbin
hit me
13:38 donsizemore
it's on my list to troubleshoot, so i haven't even done a search for these errors yet, but Google Dataset Search crabbed about the sitemap I submitted on Friday: Missing field "name" Missing field "description" Invalid object type for field "license" Invalid object type for field "creator"
13:40 pdurbin
Huh. I thought the sitemap got submitted to Google proper, not Dataset Search specifically.
13:40 donsizemore
i didn't know who owned you all's' $Goog account and whether they had seen these same errors. may post to dataverse-users
13:40 donsizemore
whoops Google Search Console (though we submitted it with Dataset Search in mind)
13:40 pdurbin
ok
13:41 pdurbin
I'm looking at what I wrote at http://guides.dataverse.org/en/4.16/installation/config.html#creating-a-sitemap-and-submitting-it-to-search-engines
13:41 donsizemore
yeah, all that part was straightforward
13:42 pdurbin
I didn't actually do these steps for Harvard Dataverse. And at the moment there is nothing at https://dataverse.harvard.edu/sitemap.xml . So I wonder if we'll see similar errors if we try again.
13:42 donsizemore
and things look good at https://dataverse.unc.edu/sitemap.xml
13:42 donsizemore
okay. maybe i'll ask dataverse-users? (or if it's a design thing at this point, i can open an issue)
13:42 pdurbin
dataverse-community. Sure, please go ahead.
13:42 donsizemore
i was just curious if your google search console owner(s) had seen the same thing
13:43 pdurbin
When I go to https://search.google.com/search-console/sitemaps?resource_id=https%3A%2F%2Fdataverse.harvard.edu%2F I'm prompted to add a sitemap.
13:43 pdurbin
Oh, interesting. It looks like we have a sitemap at alternate path.
13:44 pdurbin
I didn't set this up. I'm not sure if it's supposed to be a secret or what. :)
13:44 pdurbin
I've been thinking we didn't have a sitemap set up all all.
13:45 pdurbin
at* all
13:48 donsizemore
it can be at any path i suppose, i just used dataverse's default location
13:49 pdurbin
sure
13:54 poikilotherm joined #dataverse
13:55 poikilotherm
Hey guys, I'm back :-)
13:56 poikilotherm
pdurbin: aye, scope is null. I bet you saw that it is null on develop... ;-)
13:56 poikilotherm
And a lot of other fields are null, too
14:00 pdurbin
develop is broken? did you test it?
14:00 poikilotherm
It works
14:00 poikilotherm
But the scope is not saved to database
14:00 pdurbin
oh
14:00 poikilotherm
This doesn't seem to hurt
14:01 pdurbin
good :)
14:01 poikilotherm
But it is a bug
14:01 pdurbin
I see.
14:01 poikilotherm
And I came across it :-D
14:01 poikilotherm
Because after upgrading, it bit me
14:01 pdurbin
Are you sure we need it?
14:10 poikilotherm
Actually I think about removing it altogether
14:10 poikilotherm
The scope is always used from the provider implementations
14:11 poikilotherm
It's just a waste of database storage
14:11 pdurbin
I have some vague memory that it's used by ORCID login.
14:13 poikilotherm
Nope.
14:13 poikilotherm
Usage report: none.
14:13 poikilotherm
Just some getters and setters.
14:14 poikilotherm
Even in develop :-D
14:14 poikilotherm
Seems like a good idea to remove it
14:14 pdurbin
Just don't break anything. And make sure to communicate the change with QA.
14:15 poikilotherm
Sure.
14:15 pdurbin
You'll need to write a Flyway script.
14:15 poikilotherm
I'll need a DB migration anyway
14:15 poikilotherm
Aye :-D
14:15 pdurbin
ok
14:15 poikilotherm
We share the same mindset ;-)
16:00 pdurbin
good
16:05 pmauduit
hi pdurbin, do you know if Vyacheslav Tykhonov (4tikhonov on github) is around sometimes on IRC ?
16:06 pmauduit
I'd need some help to figure out how the dataverse docker image is bootstrapped at first run in the dataverse-docker compo
16:20 pdurbin
pmauduit: I keep inviting him here but he's more of a Skype person. He goes by Slava, by the way.
16:20 pmauduit
ok thanks
16:21 pmauduit
we can begin to talk on github actually, he already replied to my issue :)
16:21 pdurbin
Sounds good. I can try to help too. And we can ask poikilotherm when he comes back. He talks to Slava a lot.
16:23 pmauduit
my current issue is that the dataverse container tries itself to connect as postgres superuser to create its own db, but this one should already exist in the postgres container anyway
16:24 pmauduit
this is managed in an 'install' perl script
16:24 pmauduit
but I think we can bypass the psql -U postgres CREATE DATABASE step
16:24 pdurbin
You're saying that the postgres container should already have the database schema in place?
16:26 pmauduit
no, but the database should already exist
16:27 pdurbin
empty database, no schema
16:27 pdurbin
The Perl script is in the main repo so if we should add a flag for it to behave differently, please create an issue at https://github.com/IQSS/dataverse/issues
16:37 pmauduit
ok, my goal is to try to avoid the create database step, as it already exist, then we probably won't need admin access to the postgres from the dataverse container
16:48 Slava1 joined #dataverse
18:21 pdurbin
pmauduit: it looks like a flag called "skipdatabasesetup" was added in https://github.com/IQSS/dataverse/commit/6383c88 . Does that help?
18:24 donsizemore
knock knock @pdurbin?
18:24 pdurbin
donsizemore: talk to me
18:26 donsizemore
akio is about to merge his TRSA branch with develop, and i'm poking around in the Phoenix Jenkins
18:26 donsizemore
we're really just making sure we're good WRT the recent solr config changes (which I think we are)
18:26 donsizemore
also, test out the ansible/ec2 #102 branch whenever you'd like
18:27 pdurbin
awesome, I'll try the 102 branch, thanks!
18:38 pdurbin
donsizemore: progress! I forgot to turn off data explorer though.
18:39 donsizemore
yeah, separate issue for those
18:39 pdurbin
small chunks
18:39 donsizemore
but if you're happy with 102, i'll doctor up external tools next
18:40 pdurbin
I think I'm happy! Checking a couple things. I only tested "develop". Should I test "release"?
18:44 donsizemore
i did on friday but go ahead
18:45 pdurbin
testing now. Can I just merge this?
18:46 donsizemore
lemme remove one debugging line
18:46 pdurbin
ok
18:47 donsizemore
done
18:47 pdurbin
almost done spinning up "release"
18:47 pdurbin
We'll have to redo this once 4.17 is out, right?
18:48 pdurbin
uh oh
18:48 donsizemore
i just bump the version group_var
18:48 pdurbin
oh, maybe it's the debug you added
18:48 pdurbin
TASK [dataverse : debug solr_files list] ***************************************
18:48 pdurbin
fatal: [localhost]: FAILED! => {"msg": "'schema_files' is undefined"}
18:49 donsizemore
it gets set as a fact, but i'm not using it now
18:50 donsizemore
do a pull and try one more time?
18:52 pdurbin
can do
18:54 pdurbin
Well, I don't pull. I pass the ansible branch with -a to the spin up script. :)
19:03 pdurbin
donsizemore: dataverse-6142-solr-schema-changes is looking good. Want me to merge it? Do you want to go ahead?
19:03 donsizemore
i'm re-running a 'release' test in vagrant now that you scared me, but if you're happy go ahead
19:03 donsizemore
then i'll doctor up the externaltool templates
19:05 pdurbin
merged! thanks!
19:07 pdurbin
Can we talk about https://github.com/IQSS/dataverse-ansible/issues/65 too?
19:07 donsizemore
sure thing
19:09 pdurbin
So I think what I want for now if for the ec2 instance to be ready to run the API test suite (has a burrito, etc) but not actually run the test suite when I spin it up. Does that make sense?
19:09 pdurbin
is*
19:17 pdurbin
On my laptop I'm always set up this way. I always have a burrito.
19:18 pdurbin
Of course, down the road, maybe we do want the test to run on spin up as well. I don't know. But for now I think I want a little more separation, a little more control.
19:22 pdurbin
I'm trying the existing flag now (test_suite) to see what happens. :)
19:30 donsizemore
um, i think i follow you
19:31 pdurbin
it's runnign the test suite no
19:31 pdurbin
just finished actually
19:36 pdurbin
looks like test users were created but no test datasets
19:37 donsizemore
did the tests run, or did it skip them?
19:38 donsizemore
that set of tasks pulls the develop copy of run-test-suite.sh
19:40 pdurbin
some ran: [ERROR] Tests run: 46, Failures: 30, Errors: 9, Skipped: 2
19:41 pdurbin
Maybe I should back up to my actual goal. Code coverage for the API test suite. Should I start there? :)
19:46 donsizemore
what's that project management technique in which you start with the desired result, and work backward from there?
19:57 pdurbin
test driven development :)
20:00 donsizemore
i just pushed the externaltools scope fix
20:00 donsizemore
sorry the solr thing took a while, but last week wuz bizzy
20:02 pdurbin
thanks for fixing it!
20:04 pdurbin
for fixing both
20:04 pdurbin
My desired result is a report of code coverage from running the API test suite.
20:07 pdurbin
That was we can tell which code is being excercised by the API test suite.
20:07 pdurbin
And we can also identify dead code, code that is never used, in theory.
20:36 pdurbin
"message": "cause was null! User @user60f5d467 is not permitted to perform requested action."
20:37 pdurbin
I'm getting that error because of the root dataverse permissions: http://guides.dataverse.org/en/4.16/developers/testing.html#root-dataverse-permissions
20:38 pdurbin
donsizemore: I'm heading out soon but maybe we can pick this up tomorrow?
21:15 donsizemore
@pdurbin just back from the gym
21:36 pdurbin
I just got home. Tomorrow's another day. :)
21:41 donsizemore
@pdurbin i pushed what may be a fix but am growing tired of waiting on the output to test it... should've redirected first
21:41 donsizemore
have a great evening
