Time
S
Nick
Message
01:44 bjonnh joined #dataverse
07:11 Virgile joined #dataverse
09:55 juancorr joined #dataverse
10:47 poikilotherm2 joined #dataverse
11:06 GitterIntegratio joined #dataverse
12:12 donsizemore joined #dataverse
13:41 pkiraly joined #dataverse
13:49 pkiraly
Hi, I run integration tests locacally, and it seems that several ones are broken.
13:49 pkiraly
[ERROR] Tests run: 146, Failures: 26, Errors: 3, Skipped: 8
13:56 poikilotherm2
Hi pkiraly :-)
13:56 poikilotherm2
Are you sure you are running the IT tests? I see there are skipped tests...
13:57 poikilotherm2
And can you post a list of failing tests?
13:57 poikilotherm2
That would be really helpful to see what happenes
14:01 donsizemore
Jenkins seems to skip 15 these days https://jenkins.dataverse.org/job/IQSS-dataverse-develop/728/testReport/
14:12 pkiraly
I run the procedure described in conf/docker-aio/readme.md. The skipped tests must be defined in the scripts mentioned in that file.
14:47 pdurbin
pkiraly: you might want to also look at https://guides.dataverse.org/en/5.3/developers/testing.html#getting-set-up-to-run-rest-assured-tests
14:50 poikilotherm2
Morning pdurbin :-)
14:51 poikilotherm2
I'm getting my hands dirty in Solr code...
14:51 pdurbin
nice!
14:51 pdurbin
Java everywhere
14:52 poikilotherm2
I do have some questions about the code you wrote back in the day...
14:52 poikilotherm2
:-D
14:52 pdurbin
uh oh
14:53 pdurbin
Any code I wrote more than a year ago is terrible.
14:53 poikilotherm2
Oh that's just 1 week for me. 1 year is pretty good :-D
14:54 pdurbin
:)
14:54 poikilotherm2
May I hit you with this or is now not a good time?
14:59 pdurbin
I have 2 hours of meetings right now. Then I need to feed my kids lunch. You can put some questions here and I can try to answer but I'll be slow.
15:01 poikilotherm2
OK
15:01 poikilotherm2
I'll be off in ~1h so I'll be slow then, too :-D
15:01 poikilotherm2
Here comes...
15:02 poikilotherm2
1) Looking at api.Index.getSolrSchema(), I see that you fetch the fields of search.SearchFields. Looking at real world responses e. g. from Jülich DATA reveal that these fields are never included in the output...
15:03 poikilotherm2
Now I think about moving the stuff in SearchFields into a proper enum, using search.SolrField as values, feeding the data back
15:04 poikilotherm2
The background is: I need to get the fields iterable (just like you in the getSolrSchema()) to implement a validator if the active Solr schema is corresponding to the metadata blocks etc
15:07 poikilotherm2
This is a first step towards solving https://github.com/IQSS/dataverse/issues/5989 , as I need to solve my chicken-n-egg problem https://github.com/gdcc/dataverse-kubernetes/issues/188
15:07 poikilotherm2
This hit me a few times in production, which is bad :-(
15:08 poikilotherm2
But instead of having a script maintaining a conversion from TSV to Solr schema, better let Dataverse manage things. Single source of truth...
15:09 poikilotherm2
Validating if the schema and Dataverse are on the same page is a good first thing to do here ;-)
15:10 poikilotherm2
2. In `SolrField` I see some (very dated) TODOs. May I remove those, as we seemed to have sticked to this by now?
15:14 poikilotherm2
3. As I am writing a validator, I'll add a new bean to encapsulate this logic. In `SolrIndexServiceBean` and `IndexServiceBean` are the checks about orphans. IMHO it would make sense to move these checks to the new bean, too, so things are more streamlined. Sounds good?
15:16 poikilotherm2
Oh and of course there are the current status checks, also looking for stuff in the database, but not inside Solr. This might be better moved to that new checking bean, too... (And applying a common async infrastructure to those)
15:22 pkiraly
poikilotherm2, here you are:
15:22 pkiraly
Failures:
15:22 pkiraly
AccessIT.testDownloadMultipleFiles_NonLoggedInOpen:266 expected:<200> but was:<404>
15:22 pkiraly
AccessIT.testDownloadMultipleFiles_NonLoggedInRestricted:290 expected:<200> but was:<404>
15:22 pkiraly
AccessIT.testDownloadSingleFile:205 expected:<200> but was:<403>
15:22 pkiraly
AccessIT.testRequestAccess:512 expected:<200> but was:<400>
15:22 pkiraly
DatasetsIT.testCreateDeleteDatasetLink:1739 JSON path message doesn't match.
15:22 pkiraly
DatasetsIT.testCreatePublishDestroyDataset:401 Expected status code <200> doesn't match actual status code <401>.
15:22 pkiraly
DatasetsIT.testExcludeEmail:682 Expected status code <200> doesn't match actual status code <403>.
15:22 pkiraly
DatasetsIT.testLinkingDatasets:2016 Expected status code <200> doesn't match actual status code <403>.
15:22 pkiraly
DatasetsIT.testPrivateUrl:1051 expected:<404> but was:<200>
15:22 pkiraly
DatasetsIT.testRestrictFileExportDdi:2112 Expected status code <200> doesn't match actual status code <403>.
15:22 pkiraly
DownloadFilesIT.downloadAllFilesByVersion:111 Expected status code <200> doesn't match actual status code <401>.
15:22 pkiraly
DownloadFilesIT.downloadAllFilesRestricted:285 Expected status code <403> doesn't match actual status code <401>.
15:22 pkiraly
FilesIT.testFileMetaDataGetUpdateRoundTrip:1241 expected:<200> but was:<500>
15:22 pkiraly
FilesIT.testReplaceFileBadJson:908 Expected status code <200> doesn't match actual status code <400>.
15:22 pkiraly
FilesIT.test_006_ReplaceFileGood:361 JSON path message doesn't match.
15:22 pkiraly
FilesIT.test_008_ReplaceFileAlreadyDeleted:851 JSON path message doesn't match.
15:22 pkiraly
HarvestingServerIT.testOaiFunctionality:162 expected:<1> but was:<0>
15:22 pkiraly
InReviewWorkflowIT.testCuratorSendsCommentsToAuthor:421 JSON path data.notifications[0].type doesn't match.
15:22 pkiraly
LinkIT.testLinkedDataset:88 JSON path message doesn't match.
15:22 pkiraly
MoveIT.testMoveLinkedDataset:245 Expected status code <403> doesn't match actual status code <200>.
15:22 pkiraly
SearchIT.testDatasetThumbnail:422 JSON path data.datasetThumbnailBase64image doesn't match.
15:22 pkiraly
SearchIT.testSearchPermisions:138 JSON path data.total_count doesn't match.
15:22 pkiraly
SearchIT.testSubtreePermissions:1021 JSON path data.total_count doesn't match.
15:22 pkiraly
SwordIT.testCreateDatasetPublishDestroy:563 Expected status code <405> doesn't match actual status code <204>.
15:22 pkiraly
SwordIT.testDeleteFiles:737 expected:<2> but was:<0>
15:23 pkiraly
UsersIT.testMergeAccounts:200 expected:<200> but was:<400>
15:23 pkiraly
Errors:
15:23 pkiraly
FilesIT.testDataSizeInDataverse:1371 » IllegalArgument Cannot get property 'me...
15:23 pkiraly
FilesIT.testForceReplaceAndUpdate:629 » IllegalArgument Cannot get property 'f...
15:23 pkiraly
SearchIT.testAdditionalDatasetContent6300:264 » IllegalArgument Cannot get pro...
15:23 pkiraly
Tests run: 146, Failures: 26, Errors: 3, Skipped: 8
15:23 poikilotherm2
Oh oh. That definitely does not look good regarding integration tests...
15:23 poikilotherm2
Sorry, API tests
15:24 poikilotherm2
I have to admit I do not use the docker-aio setup for this. Long standing dream of doing this with testcontainers...
15:26 poikilotherm2
You created a PR for this, didn't you? Then Jenkins should give you a report
15:26 Virgile
Hello - 404 error - can't connect - firewall issue ?
15:27 poikilotherm2
Virgile he said he is running this with docker-aio. It would be more than unusual to see a firewall issue inside a container running all of the services inside it.
15:27 pkiraly
poikilotherm2, I've created a PR in one of the integration tests which were not broken at the first place.
15:31 Virgile
right ^^ i thought these downloads were pointing outside the container
15:32 Virgile
and thought it would be the fw of the host that would cause trouble
15:32 Virgile
but nvm ^^
15:42 pkiraly
Virgile, Do you refer to the issues in integration test that I reported?
15:47 Virgile
yes, but i think poikilotherm is right ^^
15:48 poikilotherm2
pdurbin another question:
15:48 poikilotherm2
5. I would like to make DatasetFieldType and SolrField easier coupled by refactoring and using the enum DatasetFieldType.FieldType instead of maintaining a second enum SolrField.SolrType, which creates more hassle then necessary. I hope this makes sense...
15:49 poikilotherm2
(This makes the code much more explicit in terms of how the dots get connected betwenn Solr and database)
16:01 pkiraly
pdurbin, the page you suggested says: "we recommend using the “all in one” Docker configuration described in conf/docker-aio/readme.txt in the root of the repo." It mentions readme.txt twicem however it is not readme.txt, but readme.md. Should I create a ticket and PR to fix it?
16:50 pkiraly
pdurbin, I've created a PR: https://github.com/IQSS/dataverse/pull/7674
16:58 dataverse-user joined #dataverse
16:59 dataverse-user left #dataverse
19:55 pdurbin
thanks for making that PR, Peter
21:51 poikilotherm2
Hey pdurbin... Too many detailed questions? 🙈
21:52 pdurbin
poikilotherm2: yes, and sorry, very busy today. And only 8 minutes left.
21:52 poikilotherm2
Oh oh
21:52 pdurbin
did you see my comment about web.xml?
21:53 poikilotherm2
No worries. I'll just do what I think is best and create a PR to review and comment then
21:53 poikilotherm2
Yes, just responded
21:53 pdurbin
just read your reply, thanks
21:54 pdurbin
Sounds like we should open an issue with Payara. Do you want to do that? They know you and like you. :)
21:54 poikilotherm2
Oh. Do they? I always leave load of work for 'em... 🤠
21:54 pdurbin
All I have is a Payara t-shirt from JavaOne circa 2017.
21:55 poikilotherm2
Yeah, for that DirConfigSource I would have appreciated some nice swag... They so marvelous great stuff to pose with
21:56 poikilotherm2
+have
21:58 pdurbin
I got a trophy once. It's very nice.
22:01 pdurbin
Anyway, I'm out. See you all tomorrow.
22:01 pdurbin left #dataverse
