Time
S
Nick
Message
00:22 jri joined #dataverse
02:38 jri joined #dataverse
13:38 donsizemore joined #dataverse
13:40 pdurbin
donsizemore: good morning
13:41 pameyer joined #dataverse
13:46 donsizemore
@pdurbin how are you, sir?
14:00 pdurbin
Good. Can you please take a look at https://github.com/IQSS/dataverse/issues/4764 ?
14:03 pameyer
@pdurbin - that's looking very kubernetes focused to me, unless I'm missing something
14:03 pameyer
which is very possible; I'm still getting caught up
14:14 donsizemore
@pdurbin at a glance, option 1 does seem more preferable, but as Mr. Thao points out, option 2 would be better for ansible and no project escapes backwards compatibility without some minor chafing
14:15 donsizemore
though as pete points out, i'd think you'd need postgres stood up first before glassfish config or warfile deployment
14:17 pameyer
I've got some homework to do on init containers; still confused about why it makes sense to have two approaches to solve the same problem (configmap / init container)
14:17 pameyer
but me being confused is not a new thing
14:20 cthao joined #dataverse
14:20 pdurbin
Part of why I'm roping donsizemore into the conversation is that I believe he said he's potentially interested in running the docker/kubernetes stuff is the context of testing new releases.
14:20 cthao
configmaps/secrets are made for user inputs. i.e: email, username, postgres pw etc
14:21 pameyer
@cthao - gotcha; thanks
14:21 cthao
init containers just merely populate the database instead of doing it after the container starts
14:21 pdurbin
I believe we all know each other from the community meeting last week. I hope everyone got to say hi.
14:22 pameyer
I might be wrong, but I'd thought that the dv/glassfish install needed to make persistent changes to both the glassfish image, and the postgres image
14:22 pameyer
so why not init containers for both?
14:23 pameyer
^ not doubting that there is a reason, just trying to understand it
14:23 pdurbin
Postgres certainly needs the schema for Dataverse. At this time it gets that schema when the war file is deployed because we have "create-tables" turned on at https://github.com/IQSS/dataverse/blob/v4.9/src/main/resources/META-INF/persistence.xml#L14
14:25 cthao
I think glassfish deployment of the war file should be moved to the init containers, but before deploying, we could set up the backbones for glassfish so install wouldn't take forever
14:25 pdurbin
And persistent changes to the Glassfish image are done by asadmin commands, for example.
14:26 cthao
so right now I am moving some parts like jhove, rserve, jvm drivers etc to dockerbuild
14:26 cthao
'docker build'
14:26 pdurbin
cthao: yes. We should be doing more in docker build. You're on the right track.
14:27 pdurbin
We're all agreed that it's somewhat insane to run the installer during `docker run`, right?
14:27 pameyer
I am
14:27 pameyer
agreed, I mean
14:28 cthao
agreed. since it's kinda complicated to do everything without a Postgres db setup
14:31 pdurbin
cthao: yeah. Have you push a branch yet? I'd love to see what you're up to.
14:31 donsizemore
@cthao I support all of this. and as @pdurbin points out, I'm very good at breaking things
14:32 pdurbin
For thorny problems like this I push several different branchs, one for each approach I try.
14:32 pameyer
this may not be worth the trouble, but is there a "hello world" glassfish/postgres app somewhere that might simply initial testing?
14:32 donsizemore
@cthao (in that "never subscribe to malice" sort of way)
14:32 pdurbin
donsizemore: that reminds me, 4.9 has been deployed to https://dataverse.harvard.edu
14:32 donsizemore
^^ascribe. need my third cup of coffee
14:32 donsizemore
@pdurbin excellent! i was going to ask about that
14:33 donsizemore
curious what they did for the great re-index
14:33 pameyer
@cthao - echoing @donsizemore; I'm really happy you're working on this. I have a tendency to look for potential failure points
14:33 cthao
@pameyer I have been building a docker image from 4.9 and pushing it to my dockerhub
14:35 cthao
@pameyer @donsizemore thanks. I'd appreciate anyone pointing out potential errors. makes the process more effective!
14:38 pdurbin
cthao: awesome that you're pushing images to dockerhub, but what about pushing changes to the code to a branch on GitHub?
14:39 pdurbin
That way, I can jump on your branch and see the same errors you're seeing.
14:39 pdurbin
And hopefully offer advice.
14:41 cthao
I created a branch on my fork called " 4764-openshift-install-redesign"
14:44 pdurbin
cthao: great! Have you pushed it yet? I don't see it at https://github.com/thaorell/dataverse/branches
14:46 cthao
just did
14:47 cthao
thanks for pointing that out
14:48 pdurbin
Sure. Looking now. Thanks.
14:51 pdurbin
cthao: looks like you're adding asadmin commands to `docker build` which makes sense. We need to get Glassfish configured during the build stage.
14:52 cthao
yeah. I omitted commands that involved info that could be in configmaps/secrets like smtp server
14:53 pdurbin
Ok. What's your immediate question or problem?
14:55 cthao
I haven't really tested it yet so I need to do that. my immediate plan is go to glassfish-setup.sh and remove lines that I moved to dockerfile, then test to see what breaks
14:56 donsizemore joined #dataverse
14:57 pdurbin
Well, you can't remove lines from glassfish-setup.sh. That script is called by the "install" script at scripts/installer/install
14:58 pdurbin
I mean, you can remove them as an experiment, but you can't remove them permanently. If you remove them you will observe that `vagrant up` will stop working, for example.
15:00 pdurbin
pameyer: in docker-aio should we also run the installer during `docker build` rather than `docker run`. Or would you prefer to keep things the way they are?
15:02 pameyer
pdurbin: running the installer is currently incompatable with running integration tests
15:03 pameyer
for the moment, I think it makes sense to leave docker-aio as it is. if the improvements come up with a better way to run ITs, then we can junk it
15:05 cthao
@pdurbin: I am just doing it for experiment
15:06 pdurbin
cthao: ok
15:08 pdurbin
pameyer: ok. I was kind of hoping you'd say something like, "Yes, let's try moving the running of the installer from `docker build` to `docker run` in docker-aio because the environment is way simplier, requiring only Docker rather than Minishift."
15:08 pameyer
pdurbin: init containers and configmaps appear to be kubernetes/openshift only :(
15:09 pameyer
and oddly enough, the minishift install docs don't mention centos
15:09 pameyer
does it make sense to switch docker-aio as a simpler testbed? probably.
15:10 pdurbin
I just find the minishift stuff to be way more fiddly than your docker-aio stuff.
15:12 pameyer
me too.
15:13 pameyer
but if the goal is to sort the installer/entrypoint issues for kubernetes/openshift, and the best path for that is kubernetes/openshift-specific tooling; then me making noise about that won't be useful
15:14 pdurbin
Yeah, like you said, it sounds like the tools are specific to kubernetes/openshift.
15:14 pdurbin
docker-aio's purpose in life is to run the API test suite, right?
15:14 pameyer
sorting installer/entrypoint for all the various docker stuff would be nice, but possibly more effort than it's worth
15:15 pameyer
API test suite, docker-dcm, and local dev. but main purpose is API tests
15:15 pdurbin
Ah, you use it for local dev?
15:15 pameyer
yup
15:16 pameyer
that's why there are notes on how to hook up a debugger
15:16 pdurbin
Ok. What does docker-aio have to do with docker-dcm? It's a dependency?
15:17 pameyer
yeah, docker-dcm depends on the dataverse image that docker-aio produces
15:18 pameyer
that dependency could be removed; but if there's no "standard" dataverse image I went with the lowest-complexity way
15:18 dataverse-user joined #dataverse
15:18 pameyer
I'm reasonably confident that nobody's running a DCM on docker in production
15:23 cthao joined #dataverse
15:33 pdurbin
pameyer: ok, thanks
15:33 pdurbin
dataverse-user: welcome!
15:34 pdurbin
cthao: there's a tiny bit of stuff to catch up on here: http://irclog.iq.harvard.edu/dataverse/2018-06-22
15:34 dataverse-user
Just Sherry eves-dropping.
15:35 pdurbin
dataverse-user: nice! We should teach you how to change your "nick" (nickname) so we know who you are :)
15:35 pdurbin
dataverse-user: are you on a mac or pc?
15:36 dataverse-user
I'm on a Mac
15:36 Sherry joined #dataverse
15:37 Sherry
I just re-joined as "Sherry"
15:37 dataverse-user left #dataverse
15:37 pdurbin
Sherry: much better! Since you're on a Mac, you might want to try Adium: https://github.com/IQSS/chat.dataverse.org/issues/2
15:38 Sherry
So w/ Chat, I can't go back a view earlier discussions (before I "joined")?
15:39 cthao
how is docker-aio and docker-dcm different from docker?
15:39 pdurbin
Sherry: yes, we have years and years of logs for this channel: http://irclog.iq.harvard.edu/dataverse
15:40 pdurbin
cthao: please read this "timeline of events" I posted regarding docker stuff: https://github.com/IQSS/dataverse/issues/4665#issuecomment-391926503 . Then I will be happy to answer. :)
15:42 cthao
@pdurbin: thanks
15:42 pdurbin
cthao: you're done reading it? I just fixed a typo. :)
15:45 pameyer
@Sherry - there should be a link to the chat logs
15:45 pameyer
http://irclog.iq.harvard.edu/dataverse/today
15:49 pameyer
@cthao - docker-aio / docker-dcm are also "docker done wrong" in a sense. they intentionally violate some best practices for containers (aka - things targeted at scalability) in favor of low implementation complexity
15:52 pameyer
ah - and I missed that pdurbin already posted the link to the logs :(
15:54 pdurbin
pameyer: no problem. You can also download all the logs in one file: http://dvn-vm1.hmdc.harvard.edu/data/mysql/irclog/irclog.tsv
15:54 pdurbin
If anyone is inclined to do any sentiment analysis. :)
15:55 pameyer
pdurbin: look, a use case for streaming data :)
15:55 pdurbin
:)
15:56 pdurbin
pameyer: yesterday Danny and I got a deep dive on a use case for streaming data having to do with https://en.wikipedia.org/wiki/DHIS but it's a long story.
16:21 dataverse-user56 joined #dataverse
16:21 dataverse-user56
Hello, We are evaluating repository technologies, including DSpace. I would like to install and run a test instance of version 6.x in Vagrant. I tried using vagrant-dspace (https://github.com/DSpace/vagrant-dspace/) . I already have Vagrant 2.0.1 with VirtualBox in use. After fixing SSH problems, in "vagrant-dspace" I can "vagrant up" and vagrant ssh. However, navigating to http://localhost:8080/jspui/ or http://localhost:8080
16:22 pdurbin
dataverse-user56: hi! I can help you with Vagrant for *Dataverse* but not *DSpace*. :)
16:40 pdurbin
(I didn't know DSpace also uses Vagrant.)
16:44 pdurbin
pameyer: there's really nothing dynamic in your docker-aio in terms of config, from my perspective. I always just let it use conf/docker-aio/default.config. So I'm not sure configmaps/secrets are needed for what I use it for, which is running the API test suite.
17:24 donsizemore joined #dataverse
17:30 bricas_
pdurbin: have you ever seen a 500 error when trying to view your notifications?
18:00 donsizemore
@bricas_ (pardon for butting in) but does Dataverse churn for a bit first?
18:02 donsizemore
@bricas_ i'm thinking of https://github.com/IQSS/dataverse/issues/2892
18:03 donsizemore
(though I thought the number of database calls had been reduced to try to shorten the lookup time)
18:05 pdurbin
donsizemore: I was thinking of that issue too
18:05 pdurbin
bricas: anything in server.log?
18:06 bricas
erm. lemme check.
18:09 bricas
also, i can't seem to tell from the docs what the customized header/footer/homepage templates will replace -- is there any visual guide for that?
18:11 bricas
got the person to try the notifications again, here's the log snippets: https://pastebin.com/VqvQLc9W
18:13 pdurbin
bricas: NullPointerException. Can you please create a GitHub issue?
18:14 bricas
sure thing.
18:15 pdurbin
bricas: thanks. Actually, can you please open an issue about a visual guide too? Seems like a great idea.
18:20 bricas
#4774 and #4775 created
18:20 bricas
:)
18:24 pdurbin
bricas: thanks!
18:25 pdurbin
Sherry: I added a link to your poster to https://github.com/IQSS/dataverse/issues/4775
18:30 bricas
=^.^=
18:33 pdurbin
bricas: the homepage is a replace. The header is an add. Same with the footer.
18:35 pdurbin
bricas: you could buzz through all the production installations listed at https://services.dataverse.harvard.edu/miniverse/map/installations-json to see if anyone is using those templates. I think some are.
18:37 bricas
thanks for the info -- the add/replace info helps a lot actually.
18:44 pdurbin
sure thing
18:46 donsizemore
@pdurbin seeing your response to lauland — just curious if you've seen anyone successfully change a stored e-mail address for a shib account?
18:46 donsizemore
@pdurbin i ask because back when we were testing shib logins and i manually swapped a couple e-mail addresses, it rendered dataverse inoperable
18:47 donsizemore
@pdurbin but i think the e-mail address may have been stored in more than one place back then? it's been ~2 years
18:49 pdurbin
donsizemore: you're sort of asking if http://guides.dataverse.org/en/4.9/user/account.html#convert-your-dataverse-account-to-use-your-institutional-log-in works, right? It should work. I tested it.
18:58 bikramj joined #dataverse
19:00 pameyer
css is an everywhere
19:06 pdurbin
pameyer: true! Do you have a header to show bricas?
19:10 pameyer
pdurbin: let me look
19:11 pameyer
pdurbin, bricas : turns out I do https://dv.sbgrid.org/
19:12 pdurbin
bikramj: I was just talking about you. All good things. :)
19:38 cthao joined #dataverse
20:12 juancorr72 joined #dataverse
20:30 pdurbin
Have a good weekend, everyone. I'm going camping so I hope it doesn't rain! :)
20:30 pdurbin left #dataverse
20:56 slack-irc-bot1 joined #dataverse
20:58 bikramj
hi phill, I am very new to IRC and missed your message! n Thank you. :)
20:58 bikramj
I am just testing the IRC -slack bridge at https://github.com/ekmartin/slack-irc
20:59 bikramj
and it looks like a success :), these messages are successfully going to my test slack channel
20:59 slack-irc-bot1
<bikram> This message is from Slack!
21:01 slack-irc-bot1
<bikram> Two way messages to IRC <-> Slack success.
21:04 pdurbin_m joined #dataverse
21:05 pdurbin_m
bikramj: Slack is what we were talking about! You got it working!
21:06 slack-irc-bot1
<bikram> yes Phil, the node app from above Github worked like a charm
21:06 pdurbin_m
bikramj: can you please open an issue at https://github.com/IQSS/dataverse/issues about setting in up?
21:06 pdurbin_m
it*
21:07 slack-irc-bot1
<bikram> Yes, I will do that. All it need is a node app running anywhere and Token from Slack.
21:08 bikramj
logging out for now, have a nice weekend :)
21:09 pameyer
pdurbin_m: enjoy the camping
21:13 pdurbin_m
pameyer: thanks. some state park on the cape. Should be fun
21:13 pdurbin_m
okay I'm really leaving now. Have a good weekend.
22:12 pameyer left #dataverse
