Time
S
Nick
Message
07:25 poikilotherm joined #dataverse
07:47 corentin95 joined #dataverse
07:49 corentin95
Hi guys, hope you're all doing fine. I'm wondering if this is the good place to ask a question if I encounter a problem with dataverse after my installation
07:49 juancorr joined #dataverse
07:49 poikilotherm
Good morning corentin95
07:50 poikilotherm
IQSS people are all at sleep, but I can try to help you
07:50 corentin95
Thanks !
07:51 corentin95
So here is my problem : I installed dataverse on a kubernetes cluster. Everything seems to be all good. I can create a user. But when I connect using the user I just created, I land on the main webpage with this message : Not Authorized - You are not authorized to view this page.
07:52 poikilotherm
Ah, then you do have reached out to the right guy. I'm the maintainer of dataverse-kubernetes
07:52 poikilotherm
Did you use my repo or did you try your own stuff?
07:53 corentin95
I used the IQSS/dataverse-kubernetes repo
07:53 poikilotherm
Nice :-)
07:53 poikilotherm
What components did you choose?
07:54 poikilotherm
Minikube? k3s? AWS? Kind?
07:54 poikilotherm
(Lots of options out there)
07:55 corentin95
We have a cluster on Exoscale (Swiss cloud provider), this is where I deployed it using IQSS/dataverse-kubernetes/tree/master/k8s
07:56 poikilotherm
Ok. What K8s version is Exoscale using?
07:57 corentin95
let me check that :-)
07:59 corentin95
v1.12.3
07:59 poikilotherm
Ui. That might be a problem
07:59 poikilotherm
I am testing all of this stuff on 1.14
08:00 poikilotherm
I'd be happy to help debugging this.
08:00 poikilotherm
Would you like me to take a look at your deployment?
08:01 corentin95
Yes sure !
08:01 poikilotherm
Any chance you can grant me temporary access to take a look?
08:01 poikilotherm
Would very much ease debugging ;-)
08:02 poikilotherm
We can do a live video meeting to look at it together
08:02 corentin95
Yes I know ! :-) I'll have to ask my superior before due to security issue ...
08:03 poikilotherm
Sure. I dunno which timezone you're in, but I'll be around for a few hours now.
08:03 poikilotherm
Its 10:00 here.
08:04 corentin95
I'm glad to here that, same here !
08:04 poikilotherm
Where are you from?
08:04 stefankasberger joined #dataverse
08:04 corentin95
Switzerland, french part
08:05 poikilotherm
Ah, ok :-)
08:05 poikilotherm
This is where I am now... https://www.openstreetmap.org/node/367648305
08:06 corentin95
That's a bit far away :-D
08:06 poikilotherm
At least same timezone :-D
08:07 poikilotherm
Phil Durbin from IQSS will be around in about 2 hours
08:07 poikilotherm
He's an early bird
08:07 poikilotherm
I'm sure he will love to hear more from what you guys are up to, etc
08:08 poikilotherm
Pretty sure there is no swiss based installation on the map yet
08:08 poikilotherm
He is always eager to see new repos on that map ;-)
08:09 corentin95
Great to hear ! :-)
08:09 corentin95
And yeah I'm fortunate it's the same timezone
08:11 corentin95
Unfortunatly I'm afraid I won't be able to grand you a temporary access for the moment :'(
08:13 poikilotherm
Meh
08:14 poikilotherm
Alright then.
08:14 poikilotherm
How did you deploy?
08:14 poikilotherm
kubectl apply -k ?
08:15 corentin95
Sorry...
08:15 corentin95
Yes
08:15 corentin95
I followed this workflow :
08:15 corentin95
https://camo.githubusercontent[…]d776f726b732e6d64
08:16 poikilotherm
Hehehe. You should be aware that this is the internal workflow and normally hidden from you, as the jobs do all the hard work
08:17 corentin95
Hmmmm, so I should have deploy in a different way ?
08:17 poikilotherm
That depends on what you actually did.
08:17 poikilotherm
When you tried to do everything by yourself, there might be sth missing.
08:17 corentin95
So, let me describe it to you
08:17 corentin95
Yes !
08:17 corentin95
I had to declare a storage class
08:19 corentin95
It was missing so the pvc was stuck in a pending state
08:19 corentin95
I also added an ingress to expose dataverse but I don't thing this is the problem here
08:20 corentin95
(It's exposed using traefik)
08:22 corentin95
So basically, I should have only run "kubectl apply -f ." ?
08:22 corentin95
And maybe add this ingress
08:22 poikilotherm
Not -f, but -k. As I do not offer a helm chart or operator right now, people can use (builtin) kustomization
08:23 corentin95
Oups sorry, -k
08:23 corentin95
Should I try to redeploy that way ?
08:24 poikilotherm
You could do so. Please keep in mind that you can create you own kustomization file with the necessary patch for that storage class.
08:24 poikilotherm
You can even reference the repo remotely
08:25 poikilotherm
There are a few options for patching in Kustomization. A working example for AWS storage class which might work as a template for you is in docs/aws-demo
08:27 corentin95
Okay, thanks ! I'm going to take a look right away and keep you updated :-)
08:27 poikilotherm
We can still do a VC if you want with screen sharing.
08:27 poikilotherm
Up to you.
08:28 poikilotherm
And I would love you to open issues for what you think is missing
08:28 poikilotherm
Or even PRs adding docs about what would have helped you
08:28 corentin95
Could be nice ! Don't want to bother you too much. In the first place I'm going to take a look on my own and I'll come back to you if I got any problems.
08:29 poikilotherm
Okidoki :-)
08:29 corentin95
I'll do it once it's all fixed, be sure of that :-)
08:32 poikilotherm
Thank you!
08:37 pdurbin_m joined #dataverse
08:37 pdurbin_m
corentin95: have you published the root dataverse yet?
08:40 corentin95
Not sure ... How should I do that ?
08:41 poikilotherm
http://guides.dataverse.org/en/latest/user/dataverse-management.html#publish-your-dataverse
08:42 poikilotherm
Meh. A screenshot would be benefical for that part of the guide
08:47 corentin95 joined #dataverse
08:49 corentin95
I don't see any publish button ...
08:50 poikilotherm
You need to be logged in as dataverseAdmin
08:50 poikilotherm
(or other admin user)
08:50 poikilotherm
Otherwise it's hidden
08:56 corentin95
Oh great !! That works :-D
08:58 pdurbin_m
corentin95: fantastic :)
09:01 corentin95
I may have a couple of questions left...
09:02 * poikilotherm
is all ears
09:04 corentin95
Oh no, sorry to bother you find the answer !
09:06 corentin95
Well, thank you very much for your help. I really appreciated it ! You spare me many working hours. Thanks again :-)
09:07 poikilotherm
Sure. Again: I'd love issues and feedback on dataverse-kubernetes
09:07 poikilotherm
You might have seen my 4.16 milestone.
09:07 poikilotherm
There are some things still left open
09:07 poikilotherm
If you would like to help, go ahead.
09:07 poikilotherm
It's Hacktoberfest :-)
09:08 poikilotherm
https://hacktoberfest.digitalocean.com/
09:08 corentin95
Let me check that :-) I would gladly help if I can
09:09 pdurbin_m
corentin95: you are also welcome to help with the Installation Guide. :)
09:12 corentin95
Oh actually, I might have a final question ...
09:14 corentin95
To allow a user to add a dataset for example, does he/her has to be a superuser ?
09:15 corentin95
Because when I'm logged as a user lambda, I can't that " + Add data " button ...
09:15 poikilotherm
pdurbin_m that's a question for you :-D I out for lunch
09:15 poikilotherm
I am
09:18 corentin95
Enjoy your meal :-)
09:21 pdurbin_m
corentin95: please read http://guides.dataverse.org/en/4.16/installation/config.html#root-dataverse-permissions
09:22 corentin95
Oh yes, I just didn't find the "edit" all good now thanks :9
09:22 corentin95
:)
09:25 pdurbin_m
Sure! The guides are in the main repo on GitHub if you'd like to open an issue or fix them up.
09:38 corentin95
I may open an issue about those storage class, I add to add them myself (no big deal) but just to let people know
09:40 pdurbin_m
corentin95: please do!
09:41 corentin95
had * I will :-)
09:45 pdurbin joined #dataverse
09:45 pdurbin
ah, now I'm on a full sized keyboard :)
09:47 pdurbin
corentin95: it's pretty cool that you're using Kubernetes. One thing you should be aware of is that dataverse-kubernetes is a community project (lead by poikilotherm). No one at IQSS has started using it yet. I'm not saying there's anything wrong with it. I'm just thinking of future emails to support
09:51 pdurbin
That said I have big dreams for dataverse-kubernetes for demo purposes. :)
09:58 poikilotherm
Re
10:00 poikilotherm
pdurbin are you doing anything special for that hot reloading of JSF components in Netbeans? Or is this working because of the exploded WAR?
10:01 poikilotherm
I was looking for options to lower cycle times...
10:02 pdurbin
Are you talking about before or after https://github.com/IQSS/dataverse/issues/6208 ? I can talk about a year ago or I can talk about now.
10:04 poikilotherm
Dunno. It's not working from Netbeans anymore? Danny mentioned a workaround
10:04 poikilotherm
I don't want to switch to Netbeans, but "know your options"
10:04 pdurbin
The work around is to revert https://github.com/IQSS/dataverse/commit/5f99037 locally but not commit it. Not a great workaround.
10:05 pdurbin
You're on intellij, right?
10:05 poikilotherm
Aye
10:05 poikilotherm
There is of course JDWP. But this is limited to classes and even with that limited to no scope or signature changes
10:06 poikilotherm
There is HotSwap + DCEVM, but this seems like a stalled project
10:06 poikilotherm
No updates for DCEVM anymore :-(
10:06 poikilotherm
And there is JRebel
10:06 pdurbin
Is there something like http://guides.dataverse.org/en/4.16/developers/tools.html#netbeans-connector-chrome-extension for intellij?
10:07 poikilotherm
Hmm I dunno
10:07 poikilotherm
Many people use LiveReload these dasys
10:07 pdurbin
link?
10:07 poikilotherm
http://livereload.com/
10:08 poikilotherm
But you need support in the app for that
10:08 poikilotherm
It's in Spring Boot for example
10:08 pdurbin
this is what the Netbeans Connector Chrome Extension gives you
10:10 pdurbin
hot reloading
10:10 poikilotherm
Yeah
10:11 poikilotherm
LiveReload uses a Firefox/Chrome/... plugin
10:11 poikilotherm
Connecting to a server side that calls out for reload
10:11 poikilotherm
Some projects (like Spring) integrated the server part in their dev tools
10:12 poikilotherm
Looks like JRebel does it, too
10:14 pdurbin
There's a nice podcast about JSF and related technologies: http://enterprisejavanews.com/episode-41-on-java-ee-guardians-jakarta-ee-vs-microprofile-jdk-13-and-javafx
10:16 poikilotherm
Ui. 1:16. Going to listen to that while driving :-)
10:16 poikilotherm
Thx for the suggestion
10:16 poikilotherm
Reads promising
10:17 pdurbin
Oh, and there's another UI thing I wanted to show you. One sec.
10:17 pdurbin
Lightning Web Components: https://developer.salesforce.com/blogs/2018/12/introducing-lightning-web-components.html
10:18 pdurbin
"Many features that required frameworks now come standard: you no longer need a proprietary component model, proprietary language extensions, proprietary modules, etc."
10:18 pdurbin
"Lightning Web Components is the Salesforce implementation of that new breed of lightweight frameworks built on web standards."
10:19 poikilotherm
:-)
10:20 pdurbin
What do you think?
10:23 poikilotherm
It's a pretty good idea to use Standards. I don't know how mature these are and how mature the implementations are. And I dunno if they are widespread enough. I remember to old days when IE did not support CSS transparency and you had to hack around that.
10:29 pdurbin
Sure. I'm thinking about the future. Maybe five years out. For now I think we should probably use React.
10:29 poikilotherm
My colleague Dirk know a lot about this...
10:29 poikilotherm
He's using Vue for the open access monitor
10:30 pdurbin
Ok, let's use Vue.
10:30 poikilotherm
This is all frameworks anyway
10:30 pdurbin
yeah
10:31 poikilotherm
Setting up a project for such a transition is going to take a while.
10:31 pdurbin
yeah
10:31 poikilotherm
It's also likely that the project itself will take some time
10:31 poikilotherm
So maybe that's enough time to have sufficient support for things out in the wild?
10:31 pdurbin
JSF -> React/Vue -> Web Components
10:31 poikilotherm
So a transition to web components now might be a good idea
10:31 pdurbin
yeah
10:32 poikilotherm
I'm pretty sure this will take at least half a year to get this into the pipeline, funding etc. Coding for a year. That's a lot of water running down the rivers
10:33 poikilotherm
And maybe you will simple have to say "we only support browsers XXX"
10:33 poikilotherm
And no IE9, IE11, ...
10:33 pdurbin
sure, evergreen browsers only
10:34 poikilotherm
Everybody is welcome to write its own UI for older systems
10:34 poikilotherm
There is a well defined API
10:34 pdurbin
yeah
10:35 pdurbin
Dataverse can have multiple UIs. Just like Jenkins.
10:35 pdurbin
Kito is into Web Components. You'll hear him talk about them in that podcast above.
10:38 jri joined #dataverse
11:09 yoh joined #dataverse
11:29 yoh joined #dataverse
12:03 dataverse-user joined #dataverse
12:03 dataverse-user
hi for all
12:04 dataverse-user
i have a little error with worldmap, when I try to configure that
12:04 dataverse-user
error "gis_data_info" not available
12:04 dataverse-user
Sorry! This request is not from a Dataverse registered to use the WorldMap. This dataverse url was not recognized: http://research-data.urosario.edu.co/file.xhtml?fileId=15&version=1.0
12:10 pdurbin_m joined #dataverse
12:11 pdurbin_m
dataverse-user: hi! Please see DATAVERSE_SERVER_URL at https://github.com/IQSS/geoconnect/blob/master/heroku_setup.md#configure-additional-settings
12:14 corentin95 joined #dataverse
12:47 pdurbin
dataverse-user: still there?
13:15 pdurbin
poikilotherm: I just heard about another Web Components framework called ui5 ( https://github.com/SAP/ui5-webcomponents ) at http://adambien.blog/roller/abien/entry/sap_odata_opensource_and_apache (airhacks #55 at http://airhacks.fm )
13:16 pdurbin
Which one sounds more promising? Lightning Web Components or UI5 Web Components?
13:22 poikilotherm
I have absolutely no idea :-D
13:22 poikilotherm
Let me listen to the podcasts first, ok?
13:22 pdurbin
sure, sounds good, but it's only mentioned in passing :)
13:24 pdurbin
the more interesting part of that episode is https://olingo.apache.org
13:26 poikilotherm
Oh, ODATA
13:26 poikilotherm
open-access-monitor has been using it.
13:26 pdurbin
yeah?
13:26 poikilotherm
But not from Apache
13:26 poikilotherm
It was an MS implementation in C#
13:27 poikilotherm
And IIRC they quited using it because they switched to MongoDB as datastore
13:28 pdurbin
on the podcast they made it sound like you code the front end to ODATA
13:29 donsizemore joined #dataverse
13:37 pdurbin
donsizemore: mornin'. I just opened https://github.com/scholarsportal/Dataverse-Data-Curation-Tool/issues/8
13:57 pdurbin
poikilotherm: almost time for the call, right?
13:57 poikilotherm
Yeah
14:00 * poikilotherm
waits for the meeting to start...
14:00 pdurbin
"The meeting will begin when Dataverse Team arrives"
14:01 pdurbin
Here's how to connect if anyone here is interested: https://github.com/IQSS/dataverse/pull/6192#issuecomment-534830250
14:01 poikilotherm
"Now approaching sector 6, anomalous materials"
14:06 donsizemore
@pdurbin oh, i was just going to add it back in once localeCode arrived in release
14:07 pdurbin
donsizemore: cool
14:07 pdurbin
poikilotherm: should we just take notes here?
14:08 pdurbin
"what type of microsoft logins are supported?"
14:08 dataverse-user
pdurbin yes im here
14:10 pdurbin
alemania
14:10 pdurbin
"suggested vocabularies"
14:11 dataverse-user joined #dataverse
14:12 pdurbin
dataverse-user: hi! poikilotherm and I are on a call right now. For another 45 minutes or so.
14:12 dataverse-user
excuseme, i lost connection
14:13 pdurbin
dataverse-user: you are probably suffering from this bug: https://github.com/IQSS/chat.dataverse.org/issues/3 . Not your fault. You might want to try http://webchat.freenode.net/?channels=#dataverse instead.
14:14 pdurbin
poikilotherm: https://datatest.servehttp.com
14:15 hblancoca joined #dataverse
14:15 hblancoca
hi
14:17 hblancoca
"error "gis_data_info" not available" "Sorry! This request is not from a Dataverse registered to use the WorldMap" i believe that my dataverse research-datageo.urosario.edu.co is not registeded
14:22 dataverse-user joined #dataverse
14:23 poikilotherm
pdurbin: I will have to leave in ~20min to pick up kids...
14:24 pdurbin
poikilotherm: sounds like the tested with "microsoft live" accounts and whatever one calls the microsoft accounts for an institution. Two kinds of accounts.
14:24 poikilotherm
Right.
14:25 poikilotherm
You will only have local AD accounts available with an Azure AD setup that is linked with the local one.
14:25 poikilotherm
As the docs say ;-)
14:55 pdurbin
dataverse-user: ok, the call is over. Sorry about that. Still here? :)
15:29 xarthisius
pdurbin: can I ask you a favor? Could you change WT ext tool manifest on dev2.dataverse.org to point to https://girder.local.wholetale.org/api/v1/integration/dataverse ?
15:30 xarthisius
also siteUrl being returned now is https://ec2-184-73-31-204.compute-1.amazonaws.com which is midly inconvienient. Could it match FQDN of the site (dev2.dataverse.org) ?
15:31 jri joined #dataverse
15:32 jri_ joined #dataverse
15:33 jri joined #dataverse
15:44 pdurbin
xarthisius: sure! I'm in a meeting right now but will fix it up soon.
15:46 xarthisius
thanks!
16:32 pdurbin
xarthisius: can you please give this a try? https://dev2.dataverse.org/dataset.xhtml?persistentId=doi:10.5072/FK2/VKI4P5
18:05 donsizemore
@pdurbin does https://github.com/IQSS/dataverse-ansible/issues/113 sound robust enough to move API test suiting into jenkins? terminate all instances tagged say jenkins_delete_me with cron
18:06 pdurbin
let me look at https://github.com/IQSS/dataverse-ansible/compare/113_support_ec2_tags
18:07 pdurbin
looks like a nice generic solution, good work
18:15 donsizemore
well, it's a 'for now' kind of thing.
18:15 pdurbin
sure
18:15 pdurbin
steady progress :)
18:16 pdurbin
bit by bit we'll get there
18:25 nightowl joined #dataverse
18:27 nightowl
hi all...I'm trying to install dataverse in AWS. Seems everything is geared for starting with MAC. However, I am a Windows user. Can I create the EC2 instance manually and then use the "dataverse-ansible" script/instructions to install?
18:27 nightowl
referring to this specifically: http://guides.dataverse.org/en/latest/developers/deployment.html
18:31 pdurbin
nightowl: yeah, we're very Mac and Linux oriented when it comes to dev. Sorry. You can absolutely spin up an EC2 instance (or Google or Azure or whatever) any way you know how and then follow the "quickstart" I added to the dataverse-ansible README.
18:33 nightowl
it looks like it is creating security groups, key pairs, choosing the CentOS 7 AMI, t2.medium, and installing a few dependencies...right? I think we would like to use our own security groups and key pairs anyway. Thanks for the response! New to Dataverse and AWS! I think I need to get a MAC!
18:34 pdurbin
Heh. Macs are nice! I use Linux at home. :)
18:35 pdurbin
nightowl: if you're successful I might ask you to make a pull request to make the dataverse-ansible option more prominent at http://guides.dataverse.org/en/4.16/installation/prep.html#choose-your-own-installation-adventure
18:35 pdurbin
donsizemore: Leonid keeps gushing about how much he loves dataverse-ansible.
18:36 nightowl
I will do so. Thanks for the response...this is a great support option!
18:36 nightowl
I tried using the dataverse-ansible on a local VM and it crashed...I couldn't figure out where...I assume it is specific to AWS?
18:37 pdurbin
No, it should work fine in Vagrant, for example.
18:38 pdurbin
There's a Vagrantfile in there.
18:38 nightowl
oh I was just running it in a virtualbox VM ...not with Vagrant
18:48 jri joined #dataverse
18:54 donsizemore
@pdurbin it's looking like RHEL8/CentOS8 will require Postgres-10+
18:55 pdurbin
donsizemore: ok, in https://github.com/IQSS/dataverse/issues I don't see an issue for Postgres 10 support. Do you mind creating one?
18:57 donsizemore
well, either that or the postgres repos can provide an older python-psycopg2 module, but i don't see them putting much work into postgres-9.6 under rhel/centos8
18:58 pdurbin
what version does AWS RDS support?
18:59 donsizemore
@pdurbin i think you're good https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/CHAP_PostgreSQL.html#PostgreSQL .Concepts
19:00 pdurbin
I see a lot of 10.x. Good. Thanks.
19:02 donsizemore
and i have dataverse deploying on centos8 now
19:02 pdurbin
nice!
19:02 pdurbin
I always call you our pioneer. :)
19:02 donsizemore
it's compiled/intended with postgresql10 so i don't think they'll backport those libraries for earlier versions
19:02 * pdurbin
hands donsizemore a coonskin cap
19:03 donsizemore
doniel boone? i heard he took every afternoon off for tin cup libations
19:03 pdurbin
oh dear, my mom says we're related to him
19:04 donsizemore
you missed my intentional misspelling ;)
19:04 pdurbin
heh, so I did
19:05 donsizemore
so... ima leave the role defaults at 9.6, and i'll update the README to note that at present RHEL /CentOS8 and Postgres < 10 are mutually exclusive
19:06 donsizemore
p.s. if you can approve my dataverse-ansible pull request, tiny though it may be, i can start switching jenkins.dataverse.org over to running tests in EC2
19:07 Alejandra joined #dataverse
19:08 pdurbin
I'll just merge it if that's ok.
19:09 GeraFp joined #dataverse
19:09 pdurbin
GeraFp: hi!
19:10 donsizemore
@pdurbin excellent. i may take a mental health trip to the gym, then i'll start on jenkins
19:11 pdurbin
donsizemore: do it. I went this morning. Merged!
19:11 donsizemore
@pdurbin i'll open the issue about PG10+ support. they can use dataverse-ansible to test ;)
19:11 pdurbin
Alejandra: hi!
19:12 GeraFp
Hi all!! :D Good Afternoon.
19:12 Alejandra
@pdurbin Hi!
19:13 pdurbin
It was a pleasure chatting with you this morning. :)
19:13 pdurbin
donsizemore: these are the folks who are starting to use our spin up scripts and dataverse-ansible. :)
19:15 donsizemore
hello =)
19:35 pdurbin
And they've got some great stuff in the works: https://github.com/IQSS/dataverse/issues/6154 and https://github.com/IQSS/dataverse/issues/6155 and https://github.com/IQSS/dataverse/issues/6156 :)
19:49 pdurbin
xarthisius: huh, the siteUrl is still wrong on dev2. I'm bringing it down to hack on it some more.
20:03 pdurbin
oh, duh, I had siteUrl in domain.xml twice
20:05 pdurbin
xarthisius: ok, please try again: https://dev2.dataverse.org/dataset.xhtml?persistentId=doi:10.5072/FK2/VKI4P5
20:05 xarthisius
pdurbin: perfect! thank you!
20:06 pdurbin
does it work? :) or... are you unblocked at least? :)
20:09 xarthisius
it works, we'll be using it to test https://github.com/whole-tale/girder_wholetale/pull/352
20:09 xarthisius
as soon as I add description :)
20:13 pdurbin
great!
20:13 pdurbin
What did you decided to to with regard to docs? Are you going to ship manifest files at both leves? File and dataset?
20:13 pdurbin
to do*
20:20 xarthisius
I think we're more interested in dataset level, so we'll focus on that. API will support any combination of fileId, filePid, datasetPid, datasetId though
20:22 pdurbin
ok, sounds fine
20:22 pdurbin
Thanks for providing Dataverse's first explore tool at the dataset level!
20:29 xarthisius
yw! :)
21:01 pdurbin
I'm feeling a Dataverse 4.17 release soon.
