Commit graph

44 commits

Author SHA1 Message Date
Alex
761e7fdd2d
feat(server): memory includes partners assets on timeline (#7993)
* feat(server): memory includes partners assets on timeline

* remove unsued code, generate sql

* fix test

* add test
2024-03-18 14:46:52 -05:00
martin
31f7e1aca3
feat(server, web): album orders (#7819)
* feat: album orders

* fix: tests

* pr feedback

* pr feedback

* pr feedback

* fix: tests

* add comment

* pr feedback

* fix: rendering issue

* wording

* fix: order value doesn't change

---------

Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
2024-03-14 11:45:03 -05:00
Mert
ee8e8a0c0f
perf(server): optimize getByIds query (#7918)
* clean up usage

* i'm not updating all these tests

* update tests

* add indices

* add indices to entities

remove index from person entity

add to face entity

fix

* simplify query

* update sql

* missing await

* remove synchronize false
2024-03-14 01:58:09 -04:00
Mert
8dc62bd29a
fix(server): face search results not always sorted (#7839)
* order by

* update sql
2024-03-11 09:19:20 -05:00
Mert
5bd597f14b
fix(server): external library sync not working for large libraries (#7759) 2024-03-10 22:30:57 -04:00
Alex
e8fb529026
fix(server): getAllAssets doesn't return all assets (#7752)
* fix(server): getAllAssets doesn't return all assets

* try reverting

* fix: archive and remove unused method

* update sql

* remove unused code

* linting
2024-03-08 17:16:32 -06:00
Sam Holton
7a4ae7d142
feat(server,web): add force delete to immediately remove user (#7681)
* feat(server,web): add force delete to immediately remove user

* update wording on force delete confirmation

* fix force delete css

* PR feedback

* cleanup user service delete for force

* adding user status column

* some cleanup and tests

* more test fixes

* run npm run sql:generate

* chore: cleanup and websocket

* chore: linting

* userRepository.restore

* removed bad color class from delete-confirm-dialoge

* additional confirmation for user force delete

* shorten confirmation message

---------

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
2024-03-08 17:49:39 -05:00
Mert
1ec5d612fa
perf(server): use queries to refresh library assets (#7685)
* use queries instead of js

* missing await

* add mock methods

* fix test

* update sql

* linting
2024-03-06 21:23:10 -06:00
Alex
52dfe5fc92
fix(server): stack info in asset response for mobile (#7346)
* fix(server): stack info in asset response for mobile

* fix(server): getAllAssets - do not filter by stack ID

* tet(server): GET /assets stack e2e

* chore(server): fix checks

* stack asset height

---------

Co-authored-by: shenlong-tanwen <139912620+shalong-tanwen@users.noreply.github.com>
2024-03-06 05:44:56 +00:00
martin
6ab404597c
fix(server): incorrect number of assets for a person (#7602)
* fix: incorrect number of assets

* fix: tests

* pr feedback

* fix: e2e test

* fix: e2e test

* fix: e2e test

* feat: more tests
2024-03-04 18:11:54 -05:00
Jonathan Jogenfors
efa6efd200
feat(server,web): remove external path nonsense and make libraries admin-only (#7237)
* remove external path

* open-api

* make sql

* move library settings to admin panel

* Add documentation

* show external libraries only

* fix library list

* make user library settings look good

* fix test

* fix tests

* fix tests

* can pick user for library

* fix tests

* fix e2e

* chore: make sql

* Use unauth exception

* delete user library list

* cleanup

* fix e2e

* fix await lint

* chore: remove unused code

* chore: cleanup

* revert docs

* fix: is admin stuff

* table alignment

---------

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
2024-02-29 12:35:37 -06:00
martin
a2934b8830
feat(server, web): search location (#7139)
* feat: search location

* fix: tests

* feat: outclick

* location search index

* update query

* fixed query

* updated sql

* update query

* Update search.dto.ts

Co-authored-by: Daniel Dietzler <36593685+danieldietzler@users.noreply.github.com>

* coalesce

* fix: tests

* feat: add alternate names

* fix: generate sql files

* single table, add alternate names to query, cleanup

* merge main

* update sql

* pr feedback

* pr feedback

* chore: fix merge

---------

Co-authored-by: mertalev <101130780+mertalev@users.noreply.github.com>
Co-authored-by: Daniel Dietzler <36593685+danieldietzler@users.noreply.github.com>
2024-02-23 19:42:37 -05:00
martin
5c0c98473d
fix(server, web): people page (#7319)
* fix: people page

* fix: use locale

* fix: e2e

* fix: remove useless w-full

* fix: don't count people without thumbnail

* fix: es6 template string

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>

---------

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
2024-02-21 17:03:45 -05:00
Mert
eb73f6605b
fix(server): don't return archived assets by default (#7278)
* don't show archived results by default

* fix e2e

* generate sql

* set default in dto

---------

Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
2024-02-21 03:59:26 +00:00
Mert
857ec0451d
feat(server): optimize get asset query (#7176)
* faster query

* add index

* remove legacy code

* update mock

* remove unused imports

* add relations

* add stack

* formatting

* remove stack relation

* remove unused import

* increase chunk size

* generate sql

* linting

* fix typing

* formatting
2024-02-18 12:22:25 -06:00
Alex
69983ff83a
feat: enhance search (#7127)
* feat: hybrid search

* fixing normal search

* building out the query

* okla

* filters

* date

* order by date

* Remove hybrid search endpoint

* remove search hybrid endpoint

* faces query

* search for person

* search and pagination

* with exif

* with exif

* justify gallery viewer

* memory view

* Fixed userId is null

* openapi and styling

* searchdto

* lint and format

* remove term

* generate sql

* fix test

* chips

* not showing true

* pr feedback

* pr feedback

* nit name

* linting

* pr feedback

* styling

* linting
2024-02-17 11:00:55 -06:00
Mert
e334443919
feat(server, web): smart search filtering and pagination (#6525)
* initial pagination impl

* use limit + offset instead of take + skip

* wip web pagination

* working infinite scroll

* update api

* formatting

* fix rebase

* search refactor

* re-add runtime config for vector search

* fix rebase

* fixes

* useless omitBy

* unnecessary handling

* add sql decorator for `searchAssets`

* fixed search builder

* fixed sql

* remove mock method

* linting

* fixed pagination

* fixed unit tests

* formatting

* fix e2e tests

* re-flatten search builder

* refactor endpoints

* clean up dto

* refinements

* don't break everything just yet

* update openapi spec & sql

* update api

* linting

* update sql

* fixes

* optimize web code

* fix typing

* add page limit

* make limit based on asset count

* increase limit

* simpler import
2024-02-12 20:50:47 -05:00
Mert
56b0643890
feat(server)!: pgvecto.rs 0.2 and pgvector compatibility (#6785)
* basic changes

update version check

set ef_search for clip

* pgvector compatibility

Revert "pgvector compatibility"

This reverts commit 2b66a52aa4097dd27da58138c5288fd87cb9b24a.

pgvector compatibility: minimal edition

pgvector startup check

* update extension at startup

* wording

shortened vector extension variable name

* nightly docker

* fixed version checks

* update tests

add tests for updating extension

remove unnecessary check

* simplify `getRuntimeConfig`

* wording

* reindex on minor version update

* 0.2 upgrade testing

update prod compose

* acquire lock for init

* wip vector down on shutdown

* use upgrade helper

* update image tag

* refine restart check

check error message

* test reindex

testing

upstream fix

formatting

fixed reindexing

* use enum in signature

* fix tests

remove unused code

* add reindexing tests

* update to official 0.2

remove alpha from version name

* add warning test if restart required

* update test image to 0.2.0

* linting and test cleanup

* formatting

* update sql

* wording

* handle setting search path for new and existing databases

* handle new db in reindex check

* fix post-update reindexing

* get dim size

* formatting

* use vbase

* handle different db name

* update sql

* linting

* fix suggested env
2024-02-06 21:46:38 -05:00
renovate[bot]
d88ca5fb2a
chore(deps): update server (#6930)
* chore(deps): update server

* chore: generate sql

---------

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Daniel Dietzler <mail@ddietzler.dev>
2024-02-06 22:06:02 +00:00
martin
fa0913120d
feat(web,server): search people (#5703)
* feat: search peoples

* fix: responsive design

* use existing count

* generate sql file

* fix: tests

* remove visible people

* fix: merge, hide...

* use component

* fix: linter

* chore: regenerate api

* fix: change name when searching for a face

* save search

* remove duplicate

* use enums for query parameters

* fix: increase to 20 for the local search

* use constants

* simplify

* fix: number of people more visible

* fix: merge

* fix: search

* fix: loading spinner position

* pr feedback
2024-01-27 19:54:31 -05:00
Zack Pollard
25cad79657
refactor: move asset stacks to their own entity (#6353)
* feat: auto-stack burst photos

* feat: move stacks to asset stack entity

* chore: pin node version with volta in server

* chore: update e2e cases

* chore: cleanup

* feat: migrate existing stacks

---------

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
2024-01-27 18:52:14 +00:00
Jason Rasmussen
7fc4abba72
feat(server): sql access checks (#6635) 2024-01-25 10:14:38 -05:00
Daniel Dietzler
607fd39130
fix(server): only calculate quota usage for internal assets (#6556)
only calculate usage for internal assets
2024-01-21 15:48:29 -06:00
Mert
68f52818ae
feat(server): separate face clustering job (#5598)
* separate facial clustering job

* update api

* fixed some tests

* invert clustering

* hdbscan

* update api

* remove commented code

* wip dbscan

* cleanup

removed cluster endpoint

remove commented code

* fixes

updated tests

minor fixes and formatting

fixed queuing

refinements

* scale search range based on library size

* defer non-core faces

* optimizations

removed unused query option

* assign faces individually for correctness

fixed unit tests

remove unused method

* don't select face embedding

update sql

linting

fixed ml typing

* updated job mock

* paginate people query

* select face embeddings because typeorm

* fix setting face detection concurrency

* update sql

formatting

linting

* simplify logic

remove unused imports

* more specific delete signature

* more accurate typing for face stubs

* add migration

formatting

* chore: better typing

* don't select embedding by default

remove unused import

* updated sql

* use normal try/catch

* stricter concurrency typing and enforcement

* update api

* update job concurrency panel to show disabled queues

formatting

* check jobId in queueAll

fix tests

* remove outdated comment

* better facial recognition icon

* wording

wording

formatting

* fixed tests

* fix

* formatting & sql

* try to fix sql check

* more detailed description

* update sql

* formatting

* wording

* update `minFaces` description

---------

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
2024-01-18 00:08:48 -05:00
Jason Rasmussen
6f291006e4
fix(server): handle 5 digit years (#6457) 2024-01-17 16:08:38 -05:00
renovate[bot]
0350058689
fix(deps): update server (#6415)
* fix(deps): update server

* chore: fix tests

---------

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
2024-01-17 18:24:51 +00:00
Sushain Cherivirala
7fc1954e2a
fix(server): add filename search (#6394)
Fixes https://github.com/immich-app/immich/issues/5982.

There are basically three options:

1. Search `originalFileName` by dropping a file extension from the query
(if present). Lower fidelity but very easy - just a standard index &
equality.
2. Search `originalPath` by adding an index on `reverse(originalPath)`
and using `starts_with(reverse(query) + "/", reverse(originalPath)`. A
weird index & query but high fidelity.
3. Add a new generated column called `originalFileNameWithExtension` or
something. More storage, kinda jank.

TBH, I think (1) is good enough and easy to make better in the future.
For example, if I search "DSC_4242.jpg", I don't really think it matters
if "DSC_4242.mov" also shows up.

edit: There's a fourth approach that we discussed a bit in Discord and
decided we could switch to it in the future: using a GIN. The minor
issue is that Postgres doesn't tokenize paths in a useful (they're a
single token and it won't match against partial components). We can
solve that by tokenizing it ourselves. For example:

```
immich=# with vecs as (select to_tsvector('simple', array_to_string(string_to_array('upload/library/sushain/2015/2015-08-09/IMG_275.JPG', '/'), ' ')) as vec)  select * from vecs where vec @@ phraseto_tsquery('simple', array_to_string(string_to_array('library/sushain', '/'), ' '));
                                      vec
-------------------------------------------------------------------------------
 '-08':6 '-09':7 '2015':4,5 'img_275.jpg':8 'library':2 'sushain':3 'upload':1
(1 row)
```

The query is also tokenized with the 'split-by-slash-join-with-space'
strategy. This strategy results in `IMG_275.JPG`, `2015`, `sushain` and
`library/sushain` matching. But, `08` and `IMG_275` do not match. The
former is because the token is `-08` and the latter because the
`img_275.jpg` token is matched against exactly.
2024-01-15 14:40:28 -06:00
Alex
d096caccac
chore(web): quota enhancement (#6371)
* chore(web): quota enhancement

* show quota in user table

* update quota for single user ioption

* Add a note how to set unlimited storage

* fixed deletion doesn't update quota

* refactor relation

* fixed test

* re-refactor

* update sql

* fix e2e test

* Update server/src/domain/user/user.service.ts

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>

* revert e2e test

---------

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
2024-01-15 09:04:29 -06:00
cfitzw
deb1f970a8
feat(server, web): quotas (#4471)
* feat: quotas

* chore: open api

* chore: update status box and upload error message

---------

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
2024-01-12 19:43:36 -05:00
Michael Manganiello
8921278447
fix(server): Execute query in AlbumRepository.removeAsset method (#6216)
The current `removeAsset` implementation just builds the query but does
not execute it. That also seems to be the reason the `@GenerateSql`
decorator was commented out.
2024-01-06 11:56:08 -06:00
Emanuel Bennici
d1e16025cf
fix(server): Fix incorrect sorting with search (#6031)
The search results are not sorted by date causing confision.

Fixes #5916
2024-01-04 14:47:09 -06:00
Jason Rasmussen
7b40c20ea5
chore(server): time bucket sql (#6156) 2024-01-03 22:32:52 -06:00
Fynn Petersen-Frey
cc7ba3c21a
feat(server): search across own+partner assets (#5966)
* feat(server): search across own+partner assets

* generate sql

* fix sql parameter
2024-01-01 17:25:22 -05:00
martin
0ed89e61ec
fix: remove archived people from explore (#6091) 2024-01-01 11:07:42 -05:00
Mert
cc2dc12f6c
fix(server): run migrations after database checks (#5832)
* run migrations after checks

* optional migrations

* only run checks in server and e2e

* re-add migrations for microservices

* refactor

* move e2e init

* remove assert from migration

* update providers

* update microservices app service

* fixed logging

* refactored version check, added unit tests

* more version tests

* don't use mocks for sut

* refactor tests

* suggest image only if postgres is 14, 15 or 16

* review suggestions

* fixed regexp escape

* fix typing

* update migration
2023-12-21 10:06:26 -06:00
Mert
b520955d0e
fix(server): add more conditions to smart search (#5806)
* add more asset conditions

* udpate sql
2023-12-17 20:17:30 -06:00
Mert
6e7b3d6f24
fix(server): fix metadata search not working (#5800)
* don't require ml

* update e2e

* fixes

* fix e2e

* add additional conditions

* select all exif columns

* more fixes

* update sql
2023-12-17 20:16:08 -06:00
Mert
e3fa32ad23
fix(server): fix inconsistent explore queries (#5774)
* remove limits

* update sql
2023-12-17 11:04:35 -06:00
Alex
f206cb9403
chore(server): simplify search face query and better clustering (#5573)
* chore(server): simplify search face query and better clustering

* update sql

* Use correct syntax for utilizing the index

* Update sql
2023-12-08 16:26:17 -06:00
Mert
2553c54b26
fix(server): select asset face columns explicitly (#5564)
* select columns explicitly

* updated sql

* formatting
2023-12-08 12:43:35 -06:00
Jason Rasmussen
1e99ba8167
feat: use pgvecto.rs (#3605) 2023-12-08 11:15:46 -05:00
martin
7702560b12
feat(web): re-assign person faces (2) (#4949)
* feat: unassign person faces

* multiple improvements

* chore: regenerate api

* feat: improve face interactions in photos

* fix: tests

* fix: tests

* optimize

* fix: wrong assignment on complex-multiple re-assignments

* fix: thumbnails with large photos

* fix: complex reassign

* fix: don't send people with faces

* fix: person thumbnail generation

* chore: regenerate api

* add tess

* feat: face box even when zoomed

* fix: change feature photo

* feat: make the blue icon hoverable

* chore: regenerate api

* feat: use websocket

* fix: loading spinner when clicking on the done button

* fix: use the svelte way

* fix: tests

* simplify

* fix: unused vars

* fix: remove unused code

* fix: add migration

* chore: regenerate api

* ci: add unit tests

* chore: regenerate api

* feat: if a new person is created for a face and the server takes more than 15 seconds to generate the person thumbnail, don't wait for it

* reorganize

* chore: regenerate api

* feat: global edit

* pr feedback

* pr feedback

* simplify

* revert test

* fix: face generation

* fix: tests

* fix: face generation

* fix merge

* feat: search names in unmerge face selector modal

* fix: merge face selector

* simplify feature photo generation

* fix: change endpoint

* pr feedback

* chore: fix merge

* chore: fix merge

* fix: tests

* fix: edit & hide buttons

* fix: tests

* feat: show if person is hidden

* feat: rename face to person

* feat: split in new panel

* copy-paste-error

* pr feedback

* fix: feature photo

* do not leak faces

* fix: unmerge modal

* fix: merge modal event

* feat(server): remove duplicates

* fix: title for image thumbnails

* fix: disable side panel when there's no face until next PR

---------

Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
2023-12-05 09:43:15 -06:00
PyKen
1a63d3837e
fix(server): Return correct asset count in album (#5465)
* fix(server): Return correct asset count in album

* Update album.repository.sql

Add generated sql
2023-12-04 10:22:00 -06:00
Jason Rasmussen
5e55a17b2a
chore(server): sql versioning (#5346)
* chore(server): sql versioning

* chore: always add newline to end of file

* refactor: generator

* chore: pr feedback

* chore: pr feedback
2023-11-30 10:10:30 -05:00