Time
S
Nick
Message
10:50 juancorr joined #dataverse
11:32 Ricarda joined #dataverse
11:33 Ricarda
I have a non-technical question. I am describing Dataverse in a paper and would like to include a citation about the project itself and about the community. Are there recent publications you can recommend? And is there information on the number of members in the DataCite community? Any help is appreciated :)
12:03 Virgile joined #dataverse
12:09 Virgile joined #dataverse
12:33 Ricarda joined #dataverse
12:43 Virgile joined #dataverse
13:03 Virgile joined #dataverse
13:13 Virgile joined #dataverse
14:31 donsizemore joined #dataverse
15:07 pameyer joined #dataverse
15:08 pameyer
looks like I missed Ricarda , but there is a recommended publication for the Dataverse project. don't recall offhand, but pretty sure it's listed at dataverse.org
15:15 pdurbin joined #dataverse
15:17 pdurbin
poikilotherm: so what did you think of that meeting?
15:30 poikilotherm
pameyer pdurbin maybe in the not so future Dataverse would be able to eat its own dogfood by depositing it to Harvards Dataverse as a scholarly output of research software engineering.
15:31 poikilotherm
pdurbin: seems like a good kickoff to me :-)
15:32 pdurbin
yeah, me too
15:32 poikilotherm
I actually liked Slavas idea of creating a Github issue tracker for topics and meetings
15:33 pdurbin
Oh, for each of the 3 subtopics?
15:33 poikilotherm
Maybe a Github-linkable issue tracker would be easier to follow for results and output compared to a closed access google drive.
15:33 poikilotherm
Input for Dataverse TV :-D
15:34 pdurbin
Ah, good point.
15:34 pdurbin
I was glad to see people pushing for recordings of the calls.
15:34 poikilotherm
Will be on a workshop from Code Ocean in ~25 minutes
15:35 pameyer
poikilotherm - you're thinking Dataverse as a software dataset in dataverse?
15:38 poikilotherm
pameyer: aye
15:38 pdurbin
poikilotherm: do you have time to talk more about the postgres driver pull request?
15:42 poikilotherm
Depends. Workshop in 20 minutes. IIRC for 1.5h
15:42 poikilotherm
So quickly now or after that
15:43 pdurbin
Can I call you? Maybe on GDCC Slack?
15:43 poikilotherm
Oh. Workshop is only 1h
15:44 poikilotherm
Go go go
15:52 pameyer
poikilotherm - that got me wondering about which dataset in dataverse has the most authors/contributors. because dataverse itself seems to have quite a few
15:59 pdurbin
probably something from physics
16:02 pameyer
probably
17:08 poikilotherm
CodeOcean is a nice product, as is Whole Tale, etc.
17:08 poikilotherm
But still I couldn't find someone answering the hard question...
17:08 poikilotherm
How do you actually provide a guarantee for reproducibility for 10 years +
17:08 poikilotherm
When using a container, you might suffer from hardware arch changes, kernel ABI changes etc etc
17:09 poikilotherm
The best answer I got to that so far was during the DCM 2020
17:09 pdurbin
10+ years is a long time. Does it help that some of these tools are open source so you can run them yourself?
17:10 poikilotherm
That those tools are open source definitely helps.
17:10 poikilotherm
Even more important is that the research software itself uses as much open source as possible
17:11 poikilotherm
It really is a much harder task to let a propiertary from 10 years ago execute your code today than running it with open source software
17:12 poikilotherm
I remember with fear all the VMs and old hardware machines we tried to keep up and running so we could use tools that old to operate a large scientific experiment
17:13 poikilotherm
Just like with Dataverse: there is no glory or funding for maintenance in science
17:21 pdurbin
Not even in other countries? :)
17:38 pameyer
to guarantee something for 10 years, you'd really need funding for 10 years
17:39 pameyer
open source tools help to a degree, but only a bit. there's an increasing amount of open-source cuda code for example
17:40 pameyer
part of why treating research reproducibility as "can you re-execute the code" seems to be to not be aiming at the right problem
17:41 donsizemore
just use perl =)
17:43 pameyer
oddly enough, I had an issue with perl code breakling lately. same system, same codebase, all of a sudden wouldn't work. but it needed to talk to a server, and that server was using https, and decided to require use of newer ciphers than the old code could handle...
17:43 donsizemore
!!!
17:43 pameyer
... but it's easier to break stuff than to keep it working ;)
17:51 pameyer
I happy that linux is so far avoiding the code-signing/notarization/etc train. wouldn't be too shocked if expiring certs caused issues for software in the future
17:59 donsizemore
you missed the expired cert chain funsies in may?
18:00 pdurbin
is written in Perl
18:02 pameyer
I absolutely did not miss the expired cert chain "fun" in may ;) . got to figure out the build process for a tool that I hadn't known embedded the expired cert on Sun when a user reported "hey, this is broken"
18:02 pameyer
has been pretty solid
18:16 poikilotherm
pdurbin did you see https://github.com/ossf/scorecard ?
18:18 pdurbin
no but I'm nervous already
18:48 pameyer
I'm not seeing anything about "this is why criteria $x is on the list"
20:10 FernandoSantana joined #dataverse
20:10 FernandoSantana
I have a problem using the API
20:10 FernandoSantana
{ status:"error", message:"Endpoint available using API key only. Please contact the dataverse administrator"}
20:12 pdurbin
FernandoSantana: hi, let me look for that error quick.
20:12 pameyer
FernandoSantana - are you setting an API key in your request headers?
20:13 FernandoSantana
moment please
20:14 FernandoSantana
yes.
20:14 pdurbin
FernandoSantana: the error is leading me to this... you probably have :BlockedApiKey set to something. Please see https://guides.dataverse.org/en/5.1.1/installation/config.html#blockedapikey
20:14 FernandoSantana
I want to remove
20:14 FernandoSantana
version 4.11 on Dataverse.
20:15 pdurbin
Can you see the key if you do curl http://localhost:8080/api/admin/settings/:BlockedApiKey ?
20:15 FernandoSantana
{ status:"error", message:"Endpoint available using API key only. Please contact the dataverse administrator"}
20:16 FernandoSantana
i have the key
20:17 FernandoSantana
Use key with the command CURL, message below
20:17 FernandoSantana
{ status:"error", message:"Endpoint available using API key only. ..
20:19 pdurbin
If you want to remove the key you should probably change :BlockedApiPolicy from unblock-key to localhost-only. See https://guides.dataverse.org/en/5.1.1/installation/config.html#blockedapipolicy
20:20 pdurbin
Most installations run with "localhost-only" I believe. It's the default.
20:20 FernandoSantana
Ok.
20:20 FernandoSantana
moment please.
20:23 FernandoSantana
error :/
20:23 FernandoSantana
{ status:"error", message:"Endpoint available using API key only. Please contact the dataverse administrator"}
20:24 FernandoSantana
use localhost only, but the message error above
20:25 FernandoSantana
[roothttp://localhost:8080/api/admin/settings | jq . % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 111 0 111 0 0 4545 0 --:--:-- --:--:-- --:--:-- 5045 parse error: Invalid numeric literal at line 1, column 9 [root
20:26 pameyer
this seems like it might call for psql
20:26 pameyer
but I defer to pdurbin's API knowledge
20:27 pdurbin
What if you just do `curl http://localhost:8080/api/admin/settings ` and remove the `| jq`?
20:28 FernandoSantana
{ status:"error", message:"Endpoint available using API key only. Please contact the dataverse administrator"}
20:28 pdurbin
Oh, I know what that is... the error isn't valid JSON . "status" and "message" should be in quotes.
20:30 pdurbin
What if you do this (using your key): curl http://localhost:8080/api/admin/settings/:BlockedApiKey?unblock-key=YOUR_KEY_HERE
20:30 FernandoSantana
even using the quotation marks there is an error.
20:30 FernandoSantana
I will do what you mentioned above
20:31 FernandoSantana
{"status":"OK","data":{"message":"******"}}[root
20:31 FernandoSantana
used curl http://localhost:8080/api/admin/settings/:BlockedApiKey?unblock-key=*****
20:32 dataverse-user joined #dataverse
20:32 pdurbin
Ok, so, to use the API right, you have to add that unblock-key=***. So we should be able to add that to change the policy.
20:33 pdurbin
Like this: curl -X PUT -d localhost-only http://localhost:8080/api/admin/settings/:BlockedApiPolicy?unblock-key=YOUR_KEY_HERE
20:33 FernandoSantana
how to remove now not to use?
20:37 pdurbin
Did that work?
20:38 FernandoSantana
yes
20:39 FernandoSantana
however any command with curl it presents the error message of the api
20:42 pdurbin
Hmm. Maybe pameyer is right. Maybe it's time to switch to psql.
20:44 pdurbin
Once you're connected to your database, you could try this: SELECT * FROM setting;
20:45 FernandoSantana
ok
20:46 FernandoSantana
I will do this
20:49 FernandoSantana
dvndb=# SELECT * FROM setting; name | content ---------------------------------------+--------------------------------------------------------------------------- :AllowSignUp | yes :SignUpUrl | /dataverseuser.xhtml?editMode=CREATE :BlockedApiEndpoints | admin,test :SystemEmail
20:49 FernandoSantana
..
20:49 FernandoSantana
:AllowSignUp | yes
20:49 FernandoSantana
:SignUpUrl | /dataverseuser.xhtml?editMode=CREATE
20:49 FernandoSantana
:BlockedApiEndpoints | admin,test
20:50 FernandoSantana
:SystemEmail | curadoriadata
20:50 FernandoSantana
:Authority | 20.500.12196
20:50 FernandoSantana
:IdentifierGenerationStyle | sequentialNumber
20:50 FernandoSantana
:DoiProvider | FAKE
20:50 FernandoSantana
:ApiTermsOfUse | http://best-practices.dataverse.org/harvard-policies/harvard-api-tou.html
20:50 FernandoSantana
:ExcludeEmailFromExport | true
20:50 FernandoSantana
:ApplicationTermsOfUse |
20:50 FernandoSantana
:TabularIngestSizeLimit | 2000000000
20:50 FernandoSantana
:TwoRavensTabularView | true
20:50 FernandoSantana
:DatasetPublishPopupCustomText | Arquivo criado com sucesso no Repositório CIDACS
20:50 FernandoSantana
:PVMinLength | 6
20:51 FernandoSantana
:PVMaxLength | 0
20:51 FernandoSantana
:PVNumberOfConsecutiveDigitsAllowed | 4
20:51 FernandoSantana
:Protocol | hdl
20:51 FernandoSantana
:FilePIDsEnabled | true
20:51 FernandoSantana
:NavbarAboutUrl | Dataverse
20:51 FernandoSantana
:Shoulder | CIDACS/
20:51 FernandoSantana
:IndependentHandleService | true
20:51 FernandoSantana
:LoginSessionTimeout | 240 :ApplicationPrivacyPolicyUrl | htt
20:51 FernandoSantana
:ApplicationPrivacyPolicyUrl | https://dataverse.org/best-practices/harvard-dataverse-privacy-policy
20:51 FernandoSantana
:MaxFileUploadSizeInBytes | 2147483648
20:51 FernandoSantana
:ZipDownloadLimit | 20000000000
20:51 FernandoSantana
:TabularIngestSizeLimit:Rdata | 20000000000
20:52 FernandoSantana
:SolrMaxFileSizeForFullTextIndexing | 31457280000
20:52 FernandoSantana
:SolrHostColonPort | dataverse.intracidacs.org:8983
20:52 FernandoSantana
:RepositoryStorageAbstractionLayerUrl | https://dataverse.intracidacs.org
20:52 FernandoSantana
:DownloadMethods | rsal/rsync
20:52 FernandoSantana
:AllowApiTokenLookupViaApi | true
20:52 FernandoSantana
:ProvCollectionEnabled | true
20:52 FernandoSantana
:DataCaptureModuleUrl | http://localhost:5000
20:52 FernandoSantana
:UploadMethods | dcm/rsync+ssh
20:52 FernandoSantana
:BlockedApiPolicy | unblock-key
20:53 FernandoSantana
...
20:53 FernandoSantana
...
20:53 FernandoSantana
these are the select select parameters
20:53 pdurbin
That's the one. The last one. You should change :BlockedApiPolicy from unblock-key to localhost-only.
20:55 pameyer
I tend to do db changes in transactions, because I know I make typo's
20:56 FernandoSantana
how do i do it right in the bank?
20:56 FernandoSantana
I don't know from the dataverse bank
20:56 pameyer
so rather than "UPDATE setting SET content='localhost-only'" , I'd do "BEGIN;" before it
20:57 pameyer
not sure what you mean by "bank" though
20:57 FernandoSantana
database*
20:57 pameyer
ah - got it
20:57 FernandoSantana
sorry
20:58 pameyer
using "BEGIN;" starts a transaction - it doesn't change the database until a following "COMMIT;" statement. so if I make a mistake, I can abort it before breaking things
20:59 pameyer
for connecting, I'd usually use something like "psql -h localhost -U postgres dvndb" - assuming that you're on the system where the database lives, and that the user and database names match
20:59 FernandoSantana
moment please
21:00 pameyer
also, if you're going to be doing stuff to the database, it's always a good idea to have backups
21:00 pdurbin
But how do you know if you've made a mistake? You do SELECT after your UPDATE? I've never played with BEGIN and COMMIT.
21:00 FernandoSantana
i have backup
21:01 pameyer
sometimes by checking with SELECT after updates
21:02 pameyer
sometimes just by not having errors in the statements I'm trying to execute
21:03 pdurbin
gotcha
21:03 FernandoSantana
ok.
21:34 pdurbin
pameyer: looks like he's all set: https://github.com/IQSS/dataverse/issues/7404 ... the curl command seems to have worked.
21:35 dataverse-user joined #dataverse
21:50 pdurbin left #dataverse
22:20 pameyer
pdurbin: great
22:50 nightowl313 joined #dataverse
