From de5f07273b021a4a0192a35aeefc96c10c8519f7 Mon Sep 17 00:00:00 2001 From: Oscar Krause Date: Mon, 2 Jan 2023 11:38:48 +0100 Subject: [PATCH 01/47] README.md - added compatibility to official dls --- README.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/README.md b/README.md index 8a16f54..c10c931 100644 --- a/README.md +++ b/README.md @@ -2,6 +2,8 @@ Minimal Delegated License Service (DLS). +Compatibility tested with official DLS 2.0.1. + This service can be used without internet connection. Only the clients need a connection to this service on configured port. From 8c515b7f2ef19a3d83909e49512817fcd5c560fe Mon Sep 17 00:00:00 2001 From: Oscar Krause Date: Mon, 2 Jan 2023 11:39:37 +0100 Subject: [PATCH 02/47] README.md - removed links from endpoints --- README.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/README.md b/README.md index c10c931..ca54b10 100644 --- a/README.md +++ b/README.md @@ -15,27 +15,27 @@ Only the clients need a connection to this service on configured port. ## Endpoints -### [`GET /`](/) +### `GET /` Redirect to `/-/readme`. -### [`GET /status`](/status) (deprecated: use `/-/health`) +### `GET /status` (deprecated: use `/-/health`) Status endpoint, used for *healthcheck*. Shows also current version and commit hash. -### [`GET /-/health`](/-/health) +### `GET /-/health` Status endpoint, used for *healthcheck*. Shows also current version and commit hash. -### [`GET /-/readme`](/-/readme) +### `GET /-/readme` HTML rendered README.md. -### [`GET /-/docs`](/-/docs), [`GET /-/redoc`](/-/redoc) +### `GET /-/docs`, `GET /-/redoc` OpenAPI specifications rendered from `GET /-/openapi.json`. -### [`GET /-/manage`](/-/manage) +### `GET /-/manage` Shows a very basic UI to delete origins or leases. @@ -63,7 +63,7 @@ List current leases. Deletes an lease. -### `GET /client-token` +### `GET /client-token` (deprecated: use `/-/client-token`) Generate client token, (see [installation](#installation)). From 838956bdb7fb280f3ea86360861d365671a804af Mon Sep 17 00:00:00 2001 From: Oscar Krause Date: Mon, 2 Jan 2023 11:40:19 +0100 Subject: [PATCH 03/47] README.md - added '-L' parameter to curl commands to follow redirects (from deprecated endpoints) --- README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index ca54b10..590c37b 100644 --- a/README.md +++ b/README.md @@ -310,7 +310,7 @@ Successfully tested with this package versions: ## Linux ```shell -curl --insecure -X GET https:///client-token -o /etc/nvidia/ClientConfigToken/client_configuration_token.tok +curl --insecure -L -X GET https:///client-token -o /etc/nvidia/ClientConfigToken/client_configuration_token.tok service nvidia-gridd restart nvidia-smi -q | grep "License" ``` @@ -323,7 +323,7 @@ Now restart `NvContainerLocalSystem` service. **Power-Shell** ```Shell -curl.exe --insecure -X GET https:///client-token -o "C:\Program Files\NVIDIA Corporation\vGPU Licensing\ClientConfigToken\client_configuration_token_$($(Get-Date).tostring('dd-MM-yy-hh-mm-ss')).tok" +curl.exe --insecure -L -X GET https:///client-token -o "C:\Program Files\NVIDIA Corporation\vGPU Licensing\ClientConfigToken\client_configuration_token_$($(Get-Date).tostring('dd-MM-yy-hh-mm-ss')).tok" Restart-Service NVDisplay.ContainerLocalSystem 'C:\Program Files\NVIDIA Corporation\NVSMI\nvidia-smi.exe' -q | Select-String "License" ``` From 0853dd64cbd0a2be01c2c57272f6da7f8ebbbb53 Mon Sep 17 00:00:00 2001 From: Oscar Krause Date: Mon, 2 Jan 2023 14:12:15 +0100 Subject: [PATCH 04/47] README.md - added known issue for error on releasing leases on windows shutdown --- README.md | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/README.md b/README.md index 590c37b..6597c19 100644 --- a/README.md +++ b/README.md @@ -433,6 +433,19 @@ Dec 20 17:53:34 ubuntu-grid-server nvidia-gridd[10354]: License acquired success +### Error on releasing leases on shutdown + +The driver wants to release current leases on shutting down windows. This endpoint needs to be a http endpoint and +is currently not implemented. The error message looks like and safely can be ignored (since we have no license +limitation :P): + +``` +<1>:NLS initialized +<1>:License acquired successfully. (Info: 192.168.178.110, NVIDIA RTX Virtual Workstation; Expiry: 2023-3-30 23:0:22 GMT) +<0>:Failed to return license to 192.168.178.110 (Error: Generic network communication failure) +<0>:End Logging +``` + # Credits Thanks to vGPU community and all who uses this project and report bugs. From ab996bb030e7c7130ca73d8d1103e3b860392171 Mon Sep 17 00:00:00 2001 From: Oscar Krause Date: Mon, 2 Jan 2023 18:04:14 +0100 Subject: [PATCH 05/47] code styling --- app/main.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/app/main.py b/app/main.py index 63df4f4..883709d 100644 --- a/app/main.py +++ b/app/main.py @@ -69,7 +69,7 @@ async def index(): @app.get('/status', summary='* Status', description='returns current service status, version (incl. git-commit) and some variables.', deprecated=True) -async def status(request: Request): +async def status(): return JSONResponse({'status': 'up', 'version': VERSION, 'commit': COMMIT, 'debug': DEBUG}) From a3e089a3d5b6b862c67dcba2b865b3a7b31d32bc Mon Sep 17 00:00:00 2001 From: Oscar Krause Date: Mon, 2 Jan 2023 18:10:11 +0100 Subject: [PATCH 06/47] added some references --- app/main.py | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/app/main.py b/app/main.py index 883709d..ad90266 100644 --- a/app/main.py +++ b/app/main.py @@ -345,6 +345,7 @@ async def auth_v1_token(request: Request): return JSONResponse(response) +# venv/lib/python3.9/site-packages/nls_services_lease/test/test_lease_multi_controller.py # {'fulfillment_context': {'fulfillment_class_ref_list': []}, 'lease_proposal_list': [{'license_type_qualifiers': {'count': 1}, 'product': {'name': 'NVIDIA RTX Virtual Workstation'}}], 'proposal_evaluation_mode': 'ALL_OF', 'scope_ref_list': ['00112233-4455-6677-8899-aabbccddeeff']} @app.post('/leasing/v1/lessor', description='request multiple leases (borrow) for current origin') async def leasing_v1_lessor(request: Request): @@ -404,6 +405,7 @@ async def leasing_v1_lessor_lease(request: Request): return JSONResponse(response) +# venv/lib/python3.9/site-packages/nls_services_lease/test/test_lease_single_controller.py # venv/lib/python3.9/site-packages/nls_core_lease/lease_single.py @app.put('/leasing/v1/lease/{lease_ref}', description='renew a lease') async def leasing_v1_lease_renew(request: Request, lease_ref: str): @@ -431,6 +433,7 @@ async def leasing_v1_lease_renew(request: Request, lease_ref: str): return JSONResponse(response) +# venv/lib/python3.9/site-packages/nls_services_lease/test/test_lease_single_controller.py @app.delete('/leasing/v1/lease/{lease_ref}', description='release (return) a lease') async def leasing_v1_lease_delete(request: Request, lease_ref: str): token, cur_time = __get_token(request), datetime.utcnow() @@ -456,6 +459,7 @@ async def leasing_v1_lease_delete(request: Request, lease_ref: str): return JSONResponse(response) +# venv/lib/python3.9/site-packages/nls_services_lease/test/test_lease_multi_controller.py @app.delete('/leasing/v1/lessor/leases', description='release all leases') async def leasing_v1_lessor_lease_remove(request: Request): token, cur_time = __get_token(request), datetime.utcnow() From 34662e6612be51a0bdef4862a5f8fa46f2724782 Mon Sep 17 00:00:00 2001 From: Oscar Krause Date: Mon, 2 Jan 2023 18:57:41 +0100 Subject: [PATCH 07/47] implemented 'LEASE_RENEWAL_PERIOD' variable --- README.md | 31 ++++++++++++++++++------------- app/main.py | 4 ++-- 2 files changed, 20 insertions(+), 15 deletions(-) diff --git a/README.md b/README.md index 6597c19..6ea0572 100644 --- a/README.md +++ b/README.md @@ -282,20 +282,25 @@ After first success you have to replace `--issue` with `--renew`. # Configuration -| Variable | Default | Usage | -|---------------------|----------------------------------------|-------------------------------------------------------------------------------------| -| `DEBUG` | `false` | Toggles `fastapi` debug mode | -| `DLS_URL` | `localhost` | Used in client-token to tell guest driver where dls instance is reachable | -| `DLS_PORT` | `443` | Used in client-token to tell guest driver where dls instance is reachable | -| `LEASE_EXPIRE_DAYS` | `90` | Lease time in days | -| `DATABASE` | `sqlite:///db.sqlite` | See [official SQLAlchemy docs](https://docs.sqlalchemy.org/en/14/core/engines.html) | -| `CORS_ORIGINS` | `https://{DLS_URL}` | Sets `Access-Control-Allow-Origin` header (comma separated string) \* | -| `SITE_KEY_XID` | `00000000-0000-0000-0000-000000000000` | Site identification uuid | -| `INSTANCE_REF` | `00000000-0000-0000-0000-000000000000` | Instance identification uuid | -| `INSTANCE_KEY_RSA` | `/cert/instance.private.pem` | Site-wide private RSA key for singing JWTs | -| `INSTANCE_KEY_PUB` | `/cert/instance.public.pem` | Site-wide public key | +| Variable | Default | Usage | +|------------------------|----------------------------------------|------------------------------------------------------------------------------------------------------| +| `DEBUG` | `false` | Toggles `fastapi` debug mode | +| `DLS_URL` | `localhost` | Used in client-token to tell guest driver where dls instance is reachable | +| `DLS_PORT` | `443` | Used in client-token to tell guest driver where dls instance is reachable | +| `LEASE_EXPIRE_DAYS` | `90` | Lease time in days | +| `LEASE_RENEWAL_PERIOD` | `0.15` | The percentage of the lease period that must elapse before a licensed client can renew a license \*1 | +| `DATABASE` | `sqlite:///db.sqlite` | See [official SQLAlchemy docs](https://docs.sqlalchemy.org/en/14/core/engines.html) | +| `CORS_ORIGINS` | `https://{DLS_URL}` | Sets `Access-Control-Allow-Origin` header (comma separated string) \*2 | +| `SITE_KEY_XID` | `00000000-0000-0000-0000-000000000000` | Site identification uuid | +| `INSTANCE_REF` | `00000000-0000-0000-0000-000000000000` | Instance identification uuid | +| `INSTANCE_KEY_RSA` | `/cert/instance.private.pem` | Site-wide private RSA key for singing JWTs | +| `INSTANCE_KEY_PUB` | `/cert/instance.public.pem` | Site-wide public key | -\* Always use `https`, since guest-drivers only support secure connections! +\*1 For example, if the lease period is one day and the renewal period is 20%, the client attempts to renew its license +every 4.8 hours. If network connectivity is lost, the loss of connectivity is detected during license renewal and the +client has 19.2 hours in which to re-establish connectivity before its license expires. + +\*2 Always use `https`, since guest-drivers only support secure connections! # Setup (Client) diff --git a/app/main.py b/app/main.py index ad90266..8d0f5a5 100644 --- a/app/main.py +++ b/app/main.py @@ -40,6 +40,7 @@ INSTANCE_KEY_RSA = load_key(str(env('INSTANCE_KEY_RSA', join(dirname(__file__), INSTANCE_KEY_PUB = load_key(str(env('INSTANCE_KEY_PUB', join(dirname(__file__), 'cert/instance.public.pem')))) TOKEN_EXPIRE_DELTA = relativedelta(hours=1) # days=1 LEASE_EXPIRE_DELTA = relativedelta(days=int(env('LEASE_EXPIRE_DAYS', 90))) +LEASE_RENEWAL_PERIOD = float(env('LEASE_RENEWAL_PERIOD', 0.15)) CORS_ORIGINS = str(env('CORS_ORIGINS', '')).split(',') if (env('CORS_ORIGINS')) else [f'https://{DLS_URL}'] jwt_encode_key = jwk.construct(INSTANCE_KEY_RSA.export_key().decode('utf-8'), algorithm=ALGORITHMS.RS256) @@ -365,8 +366,7 @@ async def leasing_v1_lessor(request: Request): "ref": scope_ref, "created": cur_time.isoformat(), "expires": expires.isoformat(), - # The percentage of the lease period that must elapse before a licensed client can renew a license - "recommended_lease_renewal": 0.15, + "recommended_lease_renewal": LEASE_RENEWAL_PERIOD, "offline_lease": "true", "license_type": "CONCURRENT_COUNTED_SINGLE" } From 2e950ca6f4543901f9919ea94ae4ec55df7a3437 Mon Sep 17 00:00:00 2001 From: Oscar Krause Date: Mon, 2 Jan 2023 19:14:25 +0100 Subject: [PATCH 08/47] implemented '/-/config' endpoint to list runtime environment variables --- README.md | 6 +++++- app/main.py | 19 ++++++++++++++++++- test/main.py | 5 +++++ 3 files changed, 28 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 6ea0572..52caac7 100644 --- a/README.md +++ b/README.md @@ -25,7 +25,11 @@ Status endpoint, used for *healthcheck*. Shows also current version and commit h ### `GET /-/health` -Status endpoint, used for *healthcheck*. Shows also current version and commit hash. +Status endpoint, used for *healthcheck*. + +### `GET /-/config` + +Shows current runtime environment variables and their values. ### `GET /-/readme` diff --git a/app/main.py b/app/main.py index 8d0f5a5..27dbfbf 100644 --- a/app/main.py +++ b/app/main.py @@ -81,7 +81,24 @@ async def _index(): @app.get('/-/health', summary='* Health') async def _health(request: Request): - return JSONResponse({'status': 'up', 'version': VERSION, 'commit': COMMIT, 'debug': DEBUG}) + return JSONResponse({'status': 'up'}) + + +@app.get('/-/config', summary='* Config', description='returns environment variables.') +async def _config(): + return JSONResponse({ + 'VERSION': VERSION, + 'COMMIT': COMMIT, + 'DEBUG': DEBUG, + 'DLS_URL': DLS_URL, + 'DLS_PORT': DLS_PORT, + 'SITE_KEY_XID': SITE_KEY_XID, + 'INSTANCE_REF': INSTANCE_REF, + 'TOKEN_EXPIRE_DELTA': TOKEN_EXPIRE_DELTA, + 'LEASE_EXPIRE_DELTA': LEASE_EXPIRE_DELTA, + 'LEASE_RENEWAL_PERIOD': LEASE_RENEWAL_PERIOD, + 'CORS_ORIGINS': CORS_ORIGINS, + }) @app.get('/-/readme', summary='* Readme') diff --git a/test/main.py b/test/main.py index 9ef4457..8514e78 100644 --- a/test/main.py +++ b/test/main.py @@ -56,6 +56,11 @@ def test_health(): assert response.json()['status'] == 'up' +def test_config(): + response = client.get('/-/') + assert response.status_code == 200 + + def test_readme(): response = client.get('/-/readme') assert response.status_code == 200 From 4fb624333026865fcc80636f3facd3a298eda6ae Mon Sep 17 00:00:00 2001 From: Oscar Krause Date: Mon, 2 Jan 2023 19:15:48 +0100 Subject: [PATCH 09/47] removed deprecated endpoints - '/client-token' moved to '/-/client-token' - '/status' moved to '/-/health' and '/-/config' see README.md for more information --- README.md | 6 +----- app/main.py | 10 ---------- test/main.py | 10 ---------- 3 files changed, 1 insertion(+), 25 deletions(-) diff --git a/README.md b/README.md index 52caac7..3a09c6a 100644 --- a/README.md +++ b/README.md @@ -19,10 +19,6 @@ Only the clients need a connection to this service on configured port. Redirect to `/-/readme`. -### `GET /status` (deprecated: use `/-/health`) - -Status endpoint, used for *healthcheck*. Shows also current version and commit hash. - ### `GET /-/health` Status endpoint, used for *healthcheck*. @@ -67,7 +63,7 @@ List current leases. Deletes an lease. -### `GET /client-token` (deprecated: use `/-/client-token`) +### `GET /-/client-token` Generate client token, (see [installation](#installation)). diff --git a/app/main.py b/app/main.py index 27dbfbf..3708ea5 100644 --- a/app/main.py +++ b/app/main.py @@ -69,11 +69,6 @@ async def index(): return RedirectResponse('/-/readme') -@app.get('/status', summary='* Status', description='returns current service status, version (incl. git-commit) and some variables.', deprecated=True) -async def status(): - return JSONResponse({'status': 'up', 'version': VERSION, 'commit': COMMIT, 'debug': DEBUG}) - - @app.get('/-/', summary='* Index') async def _index(): return RedirectResponse('/-/readme') @@ -228,11 +223,6 @@ async def _client_token(): return response -@app.get('/client-token', summary='* Client-Token', description='creates a new messenger token for this service instance', deprecated=True) -async def client_token(): - return RedirectResponse('/-/client-token') - - # venv/lib/python3.9/site-packages/nls_services_auth/test/test_origins_controller.py # {"candidate_origin_ref":"00112233-4455-6677-8899-aabbccddeeff","environment":{"fingerprint":{"mac_address_list":["ff:ff:ff:ff:ff:ff"]},"hostname":"my-hostname","ip_address_list":["192.168.178.123","fe80::","fe80::1%enp6s18"],"guest_driver_version":"510.85.02","os_platform":"Debian GNU/Linux 11 (bullseye) 11","os_version":"11 (bullseye)"},"registration_pending":false,"update_pending":false} @app.post('/auth/v1/origin', description='find or create an origin') diff --git a/test/main.py b/test/main.py index 8514e78..980fe9e 100644 --- a/test/main.py +++ b/test/main.py @@ -44,11 +44,6 @@ def test_index(): assert response.status_code == 200 -def test_status(): - response = client.get('/status') - assert response.status_code == 200 - assert response.json()['status'] == 'up' - def test_health(): response = client.get('/-/health') @@ -76,11 +71,6 @@ def test_client_token(): assert response.status_code == 200 -def test_client_token_deprecated(): - response = client.get('/client-token') - assert response.status_code == 200 - - def test_origins(): pass From 8b934dfeef9751280637d7b9af4592e07740f512 Mon Sep 17 00:00:00 2001 From: Oscar Krause Date: Mon, 2 Jan 2023 19:23:23 +0100 Subject: [PATCH 10/47] fixed '/-/config' endpoint serialisation --- app/main.py | 22 +++++++++++----------- 1 file changed, 11 insertions(+), 11 deletions(-) diff --git a/app/main.py b/app/main.py index 3708ea5..d4da9f4 100644 --- a/app/main.py +++ b/app/main.py @@ -82,17 +82,17 @@ async def _health(request: Request): @app.get('/-/config', summary='* Config', description='returns environment variables.') async def _config(): return JSONResponse({ - 'VERSION': VERSION, - 'COMMIT': COMMIT, - 'DEBUG': DEBUG, - 'DLS_URL': DLS_URL, - 'DLS_PORT': DLS_PORT, - 'SITE_KEY_XID': SITE_KEY_XID, - 'INSTANCE_REF': INSTANCE_REF, - 'TOKEN_EXPIRE_DELTA': TOKEN_EXPIRE_DELTA, - 'LEASE_EXPIRE_DELTA': LEASE_EXPIRE_DELTA, - 'LEASE_RENEWAL_PERIOD': LEASE_RENEWAL_PERIOD, - 'CORS_ORIGINS': CORS_ORIGINS, + 'VERSION': str(VERSION), + 'COMMIT': str(COMMIT), + 'DEBUG': str(DEBUG), + 'DLS_URL': str(DLS_URL), + 'DLS_PORT': str(DLS_PORT), + 'SITE_KEY_XID': str(SITE_KEY_XID), + 'INSTANCE_REF': str(INSTANCE_REF), + 'TOKEN_EXPIRE_DELTA': str(TOKEN_EXPIRE_DELTA), + 'LEASE_EXPIRE_DELTA': str(LEASE_EXPIRE_DELTA), + 'LEASE_RENEWAL_PERIOD': str(LEASE_RENEWAL_PERIOD), + 'CORS_ORIGINS': str(CORS_ORIGINS), }) From 50e0dc8d1f7dd1c600f2945de356f8e817160c9d Mon Sep 17 00:00:00 2001 From: Oscar Krause Date: Mon, 2 Jan 2023 19:42:23 +0100 Subject: [PATCH 11/47] implemented '/leasing/v1/lessor/shutdown' for windows guests --- app/main.py | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) diff --git a/app/main.py b/app/main.py index d4da9f4..02fe569 100644 --- a/app/main.py +++ b/app/main.py @@ -487,6 +487,28 @@ async def leasing_v1_lessor_lease_remove(request: Request): return JSONResponse(response) +@app.post('/leasing/v1/lessor/shutdown', description='shutdown all leases') +async def leasing_v1_lessor_shutdown(request: Request): + j, cur_time = json.loads((await request.body()).decode('utf-8')) + + token = j['token'] + token = jwt.decode(token=token, key=jwt_decode_key, algorithms=ALGORITHMS.RS256, options={'verify_aud': False}) + origin_ref = token.get('origin_ref') + + released_lease_list = list(map(lambda x: x.lease_ref, Lease.find_by_origin_ref(db, origin_ref))) + deletions = Lease.cleanup(db, origin_ref) + logging.info(f'> [ shutdown ]: {origin_ref}: removed {deletions} leases') + + response = { + "released_lease_list": released_lease_list, + "release_failure_list": None, + "sync_timestamp": cur_time.isoformat(), + "prompts": None + } + + return JSONResponse(response) + + if __name__ == '__main__': import uvicorn From 2b8c4682708d20e1695b563baf35dca85cb3d838 Mon Sep 17 00:00:00 2001 From: Oscar Krause Date: Tue, 3 Jan 2023 07:25:09 +0100 Subject: [PATCH 12/47] main.py - fixed missing 'LEASE_RENEWAL_PERIOD' on '/auth/v1/origin' --- app/main.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/app/main.py b/app/main.py index 02fe569..e63fc5e 100644 --- a/app/main.py +++ b/app/main.py @@ -429,7 +429,7 @@ async def leasing_v1_lease_renew(request: Request, lease_ref: str): response = { "lease_ref": lease_ref, "expires": expires.isoformat(), - "recommended_lease_renewal": 0.16, + "recommended_lease_renewal": LEASE_RENEWAL_PERIOD, "offline_lease": True, "prompts": None, "sync_timestamp": cur_time.isoformat(), From 8f9d95056fa4f0bae4e616ae14e9e8d2fb764c27 Mon Sep 17 00:00:00 2001 From: Oscar Krause Date: Tue, 3 Jan 2023 09:20:18 +0100 Subject: [PATCH 13/47] code styling - migrated direct dict access to '.get()' --- app/main.py | 36 ++++++++++++++++++------------------ test/main.py | 19 +++++++++---------- 2 files changed, 27 insertions(+), 28 deletions(-) diff --git a/app/main.py b/app/main.py index e63fc5e..b4ba3fd 100644 --- a/app/main.py +++ b/app/main.py @@ -229,21 +229,21 @@ async def _client_token(): async def auth_v1_origin(request: Request): j, cur_time = json.loads((await request.body()).decode('utf-8')), datetime.utcnow() - origin_ref = j['candidate_origin_ref'] + origin_ref = j.get('candidate_origin_ref') logging.info(f'> [ origin ]: {origin_ref}: {j}') data = Origin( origin_ref=origin_ref, - hostname=j['environment']['hostname'], - guest_driver_version=j['environment']['guest_driver_version'], - os_platform=j['environment']['os_platform'], os_version=j['environment']['os_version'], + hostname=j.get('environment').get('hostname'), + guest_driver_version=j.get('environment').get('guest_driver_version'), + os_platform=j.get('environment').get('os_platform'), os_version=j.get('environment').get('os_version'), ) Origin.create_or_update(db, data) response = { "origin_ref": origin_ref, - "environment": j['environment'], + "environment": j.get('environment'), "svc_port_set_list": None, "node_url_list": None, "node_query_order": None, @@ -260,20 +260,20 @@ async def auth_v1_origin(request: Request): async def auth_v1_origin_update(request: Request): j, cur_time = json.loads((await request.body()).decode('utf-8')), datetime.utcnow() - origin_ref = j['origin_ref'] + origin_ref = j.get('origin_ref') logging.info(f'> [ update ]: {origin_ref}: {j}') data = Origin( origin_ref=origin_ref, - hostname=j['environment']['hostname'], - guest_driver_version=j['environment']['guest_driver_version'], - os_platform=j['environment']['os_platform'], os_version=j['environment']['os_version'], + hostname=j.get('environment').get('hostname'), + guest_driver_version=j.get('environment').get('guest_driver_version'), + os_platform=j.get('environment').get('os_platform'), os_version=j.get('environment').get('os_version'), ) Origin.create_or_update(db, data) response = { - "environment": j['environment'], + "environment": j.get('environment'), "prompts": None, "sync_timestamp": cur_time.isoformat() } @@ -288,7 +288,7 @@ async def auth_v1_origin_update(request: Request): async def auth_v1_code(request: Request): j, cur_time = json.loads((await request.body()).decode('utf-8')), datetime.utcnow() - origin_ref = j['origin_ref'] + origin_ref = j.get('origin_ref') logging.info(f'> [ code ]: {origin_ref}: {j}') delta = relativedelta(minutes=15) @@ -297,8 +297,8 @@ async def auth_v1_code(request: Request): payload = { 'iat': timegm(cur_time.timetuple()), 'exp': timegm(expires.timetuple()), - 'challenge': j['code_challenge'], - 'origin_ref': j['origin_ref'], + 'challenge': j.get('code_challenge'), + 'origin_ref': j.get('origin_ref'), 'key_ref': SITE_KEY_XID, 'kid': SITE_KEY_XID } @@ -320,13 +320,13 @@ async def auth_v1_code(request: Request): @app.post('/auth/v1/token', description='exchange auth code and verifier for token') async def auth_v1_token(request: Request): j, cur_time = json.loads((await request.body()).decode('utf-8')), datetime.utcnow() - payload = jwt.decode(token=j['auth_code'], key=jwt_decode_key) + payload = jwt.decode(token=j.get('auth_code'), key=jwt_decode_key) - origin_ref = payload['origin_ref'] + origin_ref = payload.get('origin_ref') logging.info(f'> [ auth ]: {origin_ref}: {j}') # validate the code challenge - if payload['challenge'] != b64enc(sha256(j['code_verifier'].encode('utf-8')).digest()).rstrip(b'=').decode('utf-8'): + if payload.get('challenge') != b64enc(sha256(j.get('code_verifier').encode('utf-8')).digest()).rstrip(b'=').decode('utf-8'): raise HTTPException(status_code=401, detail='expected challenge did not match verifier') access_expires_on = cur_time + TOKEN_EXPIRE_DELTA @@ -360,7 +360,7 @@ async def leasing_v1_lessor(request: Request): j, token, cur_time = json.loads((await request.body()).decode('utf-8')), __get_token(request), datetime.utcnow() origin_ref = token.get('origin_ref') - scope_ref_list = j['scope_ref_list'] + scope_ref_list = j.get('scope_ref_list') logging.info(f'> [ create ]: {origin_ref}: create leases for scope_ref_list {scope_ref_list}') lease_result_list = [] @@ -491,7 +491,7 @@ async def leasing_v1_lessor_lease_remove(request: Request): async def leasing_v1_lessor_shutdown(request: Request): j, cur_time = json.loads((await request.body()).decode('utf-8')) - token = j['token'] + token = j.get('token') token = jwt.decode(token=token, key=jwt_decode_key, algorithms=ALGORITHMS.RS256, options={'verify_aud': False}) origin_ref = token.get('origin_ref') diff --git a/test/main.py b/test/main.py index 980fe9e..a1e25d3 100644 --- a/test/main.py +++ b/test/main.py @@ -44,11 +44,10 @@ def test_index(): assert response.status_code == 200 - def test_health(): response = client.get('/-/health') assert response.status_code == 200 - assert response.json()['status'] == 'up' + assert response.json().get('status') == 'up' def test_config(): @@ -105,7 +104,7 @@ def test_auth_v1_origin(): response = client.post('/auth/v1/origin', json=payload) assert response.status_code == 200 - assert response.json()['origin_ref'] == ORIGIN_REF + assert response.json().get('origin_ref') == ORIGIN_REF def auth_v1_origin_update(): @@ -126,7 +125,7 @@ def auth_v1_origin_update(): response = client.post('/auth/v1/origin/update', json=payload) assert response.status_code == 200 - assert response.json()['origin_ref'] == ORIGIN_REF + assert response.json().get('origin_ref') == ORIGIN_REF def test_auth_v1_code(): @@ -138,8 +137,8 @@ def test_auth_v1_code(): response = client.post('/auth/v1/code', json=payload) assert response.status_code == 200 - payload = jwt.get_unverified_claims(token=response.json()['auth_code']) - assert payload['origin_ref'] == ORIGIN_REF + payload = jwt.get_unverified_claims(token=response.json().get('auth_code')) + assert payload.get('origin_ref') == ORIGIN_REF def test_auth_v1_token(): @@ -163,9 +162,9 @@ def test_auth_v1_token(): response = client.post('/auth/v1/token', json=payload) assert response.status_code == 200 - token = response.json()['auth_token'] + token = response.json().get('auth_token') payload = jwt.decode(token=token, key=jwt_decode_key, algorithms=ALGORITHMS.RS256, options={'verify_aud': False}) - assert payload['origin_ref'] == ORIGIN_REF + assert payload.get('origin_ref') == ORIGIN_REF def test_leasing_v1_lessor(): @@ -193,7 +192,7 @@ def test_leasing_v1_lessor_lease(): response = client.get('/leasing/v1/lessor/leases', headers={'authorization': __bearer_token(ORIGIN_REF)}) assert response.status_code == 200 - active_lease_list = response.json()['active_lease_list'] + active_lease_list = response.json().get('active_lease_list') assert len(active_lease_list) == 1 assert active_lease_list[0] == LEASE_REF @@ -218,6 +217,6 @@ def test_leasing_v1_lessor_lease_remove(): response = client.delete('/leasing/v1/lessor/leases', headers={'authorization': __bearer_token(ORIGIN_REF)}) assert response.status_code == 200 - released_lease_list = response.json()['released_lease_list'] + released_lease_list = response.json().get('released_lease_list') assert len(released_lease_list) == 1 assert released_lease_list[0] == LEASE_REF From bd5625af42a47ed1771d3b773b05c2fa961a640b Mon Sep 17 00:00:00 2001 From: Oscar Krause Date: Tue, 3 Jan 2023 13:02:37 +0100 Subject: [PATCH 14/47] main.py - removed example responses --- app/main.py | 6 +----- 1 file changed, 1 insertion(+), 5 deletions(-) diff --git a/app/main.py b/app/main.py index b4ba3fd..0fe9c2a 100644 --- a/app/main.py +++ b/app/main.py @@ -32,6 +32,7 @@ app = FastAPI(title='FastAPI-DLS', description='Minimal Delegated License Servic db = create_engine(str(env('DATABASE', 'sqlite:///db.sqlite'))) db_init(db), migrate(db) +# everything prefixed with "INSTANCE_*" is used as "SERVICE_INSTANCE_*" or "SI_*" in official dls service DLS_URL = str(env('DLS_URL', 'localhost')) DLS_PORT = int(env('DLS_PORT', '443')) SITE_KEY_XID = str(env('SITE_KEY_XID', '00000000-0000-0000-0000-000000000000')) @@ -224,7 +225,6 @@ async def _client_token(): # venv/lib/python3.9/site-packages/nls_services_auth/test/test_origins_controller.py -# {"candidate_origin_ref":"00112233-4455-6677-8899-aabbccddeeff","environment":{"fingerprint":{"mac_address_list":["ff:ff:ff:ff:ff:ff"]},"hostname":"my-hostname","ip_address_list":["192.168.178.123","fe80::","fe80::1%enp6s18"],"guest_driver_version":"510.85.02","os_platform":"Debian GNU/Linux 11 (bullseye) 11","os_version":"11 (bullseye)"},"registration_pending":false,"update_pending":false} @app.post('/auth/v1/origin', description='find or create an origin') async def auth_v1_origin(request: Request): j, cur_time = json.loads((await request.body()).decode('utf-8')), datetime.utcnow() @@ -255,7 +255,6 @@ async def auth_v1_origin(request: Request): # venv/lib/python3.9/site-packages/nls_services_auth/test/test_origins_controller.py -# { "environment" : { "guest_driver_version" : "guest_driver_version", "hostname" : "myhost", "ip_address_list" : [ "192.168.1.129" ], "os_version" : "os_version", "os_platform" : "os_platform", "fingerprint" : { "mac_address_list" : [ "e4:b9:7a:e5:7b:ff" ] }, "host_driver_version" : "host_driver_version" }, "origin_ref" : "00112233-4455-6677-8899-aabbccddeeff" } @app.post('/auth/v1/origin/update', description='update an origin evidence') async def auth_v1_origin_update(request: Request): j, cur_time = json.loads((await request.body()).decode('utf-8')), datetime.utcnow() @@ -283,7 +282,6 @@ async def auth_v1_origin_update(request: Request): # venv/lib/python3.9/site-packages/nls_services_auth/test/test_auth_controller.py # venv/lib/python3.9/site-packages/nls_core_auth/auth.py - CodeResponse -# {"code_challenge":"...","origin_ref":"00112233-4455-6677-8899-aabbccddeeff"} @app.post('/auth/v1/code', description='get an authorization code') async def auth_v1_code(request: Request): j, cur_time = json.loads((await request.body()).decode('utf-8')), datetime.utcnow() @@ -316,7 +314,6 @@ async def auth_v1_code(request: Request): # venv/lib/python3.9/site-packages/nls_services_auth/test/test_auth_controller.py # venv/lib/python3.9/site-packages/nls_core_auth/auth.py - TokenResponse -# {"auth_code":"...","code_verifier":"..."} @app.post('/auth/v1/token', description='exchange auth code and verifier for token') async def auth_v1_token(request: Request): j, cur_time = json.loads((await request.body()).decode('utf-8')), datetime.utcnow() @@ -354,7 +351,6 @@ async def auth_v1_token(request: Request): # venv/lib/python3.9/site-packages/nls_services_lease/test/test_lease_multi_controller.py -# {'fulfillment_context': {'fulfillment_class_ref_list': []}, 'lease_proposal_list': [{'license_type_qualifiers': {'count': 1}, 'product': {'name': 'NVIDIA RTX Virtual Workstation'}}], 'proposal_evaluation_mode': 'ALL_OF', 'scope_ref_list': ['00112233-4455-6677-8899-aabbccddeeff']} @app.post('/leasing/v1/lessor', description='request multiple leases (borrow) for current origin') async def leasing_v1_lessor(request: Request): j, token, cur_time = json.loads((await request.body()).decode('utf-8')), __get_token(request), datetime.utcnow() From 0e3e7cbd3aeee489cfe7d2b3d4f0b49ddc086f6a Mon Sep 17 00:00:00 2001 From: Oscar Krause Date: Tue, 3 Jan 2023 13:05:05 +0100 Subject: [PATCH 15/47] main.py - corrected leasing behaviour (migrated from 'LEASE_REF' to 'ALLOTMENT_REF') --- README.md | 3 ++- app/main.py | 14 ++++++++++---- app/orm.py | 14 +++++++++++++- doc/Database.md | 26 ++++++++++++++++++++++++++ test/main.py | 38 +++++++++++++++++++++++++------------- 5 files changed, 76 insertions(+), 19 deletions(-) create mode 100644 doc/Database.md diff --git a/README.md b/README.md index 3a09c6a..e64effa 100644 --- a/README.md +++ b/README.md @@ -292,7 +292,8 @@ After first success you have to replace `--issue` with `--renew`. | `DATABASE` | `sqlite:///db.sqlite` | See [official SQLAlchemy docs](https://docs.sqlalchemy.org/en/14/core/engines.html) | | `CORS_ORIGINS` | `https://{DLS_URL}` | Sets `Access-Control-Allow-Origin` header (comma separated string) \*2 | | `SITE_KEY_XID` | `00000000-0000-0000-0000-000000000000` | Site identification uuid | -| `INSTANCE_REF` | `00000000-0000-0000-0000-000000000000` | Instance identification uuid | +| `INSTANCE_REF` | `10000000-0000-0000-0000-000000000001` | Instance identification uuid | +| `ALLOTMENT_REF` | `20000000-0000-0000-0000-000000000001` | Allotment identification uuid | | `INSTANCE_KEY_RSA` | `/cert/instance.private.pem` | Site-wide private RSA key for singing JWTs | | `INSTANCE_KEY_PUB` | `/cert/instance.public.pem` | Site-wide public key | diff --git a/app/main.py b/app/main.py index 0fe9c2a..9259843 100644 --- a/app/main.py +++ b/app/main.py @@ -36,7 +36,8 @@ db_init(db), migrate(db) DLS_URL = str(env('DLS_URL', 'localhost')) DLS_PORT = int(env('DLS_PORT', '443')) SITE_KEY_XID = str(env('SITE_KEY_XID', '00000000-0000-0000-0000-000000000000')) -INSTANCE_REF = str(env('INSTANCE_REF', '00000000-0000-0000-0000-000000000000')) +INSTANCE_REF = str(env('INSTANCE_REF', '10000000-0000-0000-0000-000000000001')) +ALLOTMENT_REF = str(env('ALLOTMENT_REF', '20000000-0000-0000-0000-000000000001')) INSTANCE_KEY_RSA = load_key(str(env('INSTANCE_KEY_RSA', join(dirname(__file__), 'cert/instance.private.pem')))) INSTANCE_KEY_PUB = load_key(str(env('INSTANCE_KEY_PUB', join(dirname(__file__), 'cert/instance.public.pem')))) TOKEN_EXPIRE_DELTA = relativedelta(hours=1) # days=1 @@ -90,6 +91,7 @@ async def _config(): 'DLS_PORT': str(DLS_PORT), 'SITE_KEY_XID': str(SITE_KEY_XID), 'INSTANCE_REF': str(INSTANCE_REF), + 'ALLOTMENT_REF': [ALLOTMENT_REF], 'TOKEN_EXPIRE_DELTA': str(TOKEN_EXPIRE_DELTA), 'LEASE_EXPIRE_DELTA': str(LEASE_EXPIRE_DELTA), 'LEASE_RENEWAL_PERIOD': str(LEASE_RENEWAL_PERIOD), @@ -192,7 +194,7 @@ async def _client_token(): "nbf": timegm(cur_time.timetuple()), "exp": timegm(exp_time.timetuple()), "update_mode": "ABSOLUTE", - "scope_ref_list": [str(uuid4())], # this is our LEASE_REF + "scope_ref_list": [ALLOTMENT_REF], "fulfillment_class_ref_list": [], "service_instance_configuration": { "nls_service_instance_ref": INSTANCE_REF, @@ -361,12 +363,16 @@ async def leasing_v1_lessor(request: Request): lease_result_list = [] for scope_ref in scope_ref_list: + if scope_ref not in [ALLOTMENT_REF]: + raise HTTPException(status_code=500, detail=f'no service instances found for scopes: ["{scope_ref}"]') + + lease_ref = str(uuid4()) expires = cur_time + LEASE_EXPIRE_DELTA lease_result_list.append({ "ordinal": 0, # https://docs.nvidia.com/license-system/latest/nvidia-license-system-user-guide/index.html "lease": { - "ref": scope_ref, + "ref": lease_ref, "created": cur_time.isoformat(), "expires": expires.isoformat(), "recommended_lease_renewal": LEASE_RENEWAL_PERIOD, @@ -375,7 +381,7 @@ async def leasing_v1_lessor(request: Request): } }) - data = Lease(origin_ref=origin_ref, lease_ref=scope_ref, lease_created=cur_time, lease_expires=expires) + data = Lease(origin_ref=origin_ref, scope_ref=scope_ref, lease_ref=lease_ref, lease_created=cur_time, lease_expires=expires) Lease.create_or_update(db, data) response = { diff --git a/app/orm.py b/app/orm.py index 0f5d386..8128804 100644 --- a/app/orm.py +++ b/app/orm.py @@ -72,17 +72,19 @@ class Lease(Base): lease_ref = Column(CHAR(length=36), primary_key=True, nullable=False, index=True) # uuid4 origin_ref = Column(CHAR(length=36), ForeignKey(Origin.origin_ref, ondelete='CASCADE'), nullable=False, index=True) # uuid4 + scope_ref = Column(CHAR(length=36), nullable=False, index=True) # uuid4 lease_created = Column(DATETIME(), nullable=False) lease_expires = Column(DATETIME(), nullable=False) lease_updated = Column(DATETIME(), nullable=False) def __repr__(self): - return f'Lease(origin_ref={self.origin_ref}, lease_ref={self.lease_ref}, expires={self.lease_expires})' + return f'Lease(origin_ref={self.origin_ref}, scope_ref={self.scope_ref}, lease_ref={self.lease_ref}, expires={self.lease_expires})' def serialize(self) -> dict: return { 'lease_ref': self.lease_ref, 'origin_ref': self.origin_ref, + 'scope_ref': self.scope_ref, 'lease_created': self.lease_created.isoformat(), 'lease_expires': self.lease_expires.isoformat(), 'lease_updated': self.lease_updated.isoformat(), @@ -178,4 +180,14 @@ def migrate(engine: Engine): Lease.__table__.drop(bind=engine) init(engine) + def upgrade_1_2_to_1_3(): + x = db.dialect.get_columns(engine.connect(), Lease.__tablename__) + x = next((_ for _ in x if _['name'] == 'scope_ref'), None) + if x is None: + Lease.scope_ref.compile() + column_name = Lease.scope_ref.name + column_type = Lease.scope_ref.type.compile(engine.dialect) + engine.execute(f'ALTER TABLE "{Lease.__tablename__}" ADD COLUMN "{column_name}" {column_type}') + upgrade_1_0_to_1_1() + upgrade_1_2_to_1_3() diff --git a/doc/Database.md b/doc/Database.md new file mode 100644 index 0000000..5a838a3 --- /dev/null +++ b/doc/Database.md @@ -0,0 +1,26 @@ +# Database structure + +## `request_routing.service_instance` + +| xid | org_name | +|----------------------------------------|--------------------------| +| `10000000-0000-0000-0000-000000000000` | `lic-000000000000000000` | + +- `xid` is used as `SERVICE_INSTANCE_XID` + +## `request_routing.license_allotment_service_instance` + +| xid | service_instance_xid | license_allotment_xid | +|----------------------------------------|----------------------------------------|----------------------------------------| +| `90000000-0000-0000-0000-000000000001` | `10000000-0000-0000-0000-000000000000` | `80000000-0000-0000-0000-000000000001` | + +- `xid` is only a primary-key and never used as foreign-key or reference +- `license_allotment_xid` must be used to fetch `xid`'s from `request_routing.license_allotment_reference` + +## `request_routing.license_allotment_reference` + +| xid | license_allotment_xid | +|----------------------------------------|----------------------------------------| +| `20000000-0000-0000-0000-000000000001` | `80000000-0000-0000-0000-000000000001` | + +- `xid` is used as `scope_ref_list` on token request diff --git a/test/main.py b/test/main.py index a1e25d3..f04de99 100644 --- a/test/main.py +++ b/test/main.py @@ -3,7 +3,7 @@ from hashlib import sha256 from calendar import timegm from datetime import datetime from os.path import dirname, join -from uuid import uuid4 +from uuid import uuid4, UUID from dateutil.relativedelta import relativedelta from jose import jwt, jwk @@ -20,8 +20,7 @@ from app.util import load_key client = TestClient(main.app) -ORIGIN_REF, LEASE_REF = str(uuid4()), str(uuid4()) -SECRET = "HelloWorld" +ORIGIN_REF, ALLOTMENT_REF, SECRET = str(uuid4()), '20000000-0000-0000-0000-000000000001', 'HelloWorld' # INSTANCE_KEY_RSA = generate_key() # INSTANCE_KEY_PUB = INSTANCE_KEY_RSA.public_key() @@ -177,15 +176,16 @@ def test_leasing_v1_lessor(): 'product': {'name': 'NVIDIA RTX Virtual Workstation'} }], 'proposal_evaluation_mode': 'ALL_OF', - 'scope_ref_list': [LEASE_REF] + 'scope_ref_list': [ALLOTMENT_REF] } response = client.post('/leasing/v1/lessor', json=payload, headers={'authorization': __bearer_token(ORIGIN_REF)}) assert response.status_code == 200 - lease_result_list = response.json()['lease_result_list'] + lease_result_list = response.json().get('lease_result_list') assert len(lease_result_list) == 1 - assert lease_result_list[0]['lease']['ref'] == LEASE_REF + assert str(UUID(lease_result_list[0]['lease']['ref'])) == lease_result_list[0]['lease']['ref'] + return lease_result_list[0]['lease']['ref'] def test_leasing_v1_lessor_lease(): @@ -194,29 +194,41 @@ def test_leasing_v1_lessor_lease(): active_lease_list = response.json().get('active_lease_list') assert len(active_lease_list) == 1 - assert active_lease_list[0] == LEASE_REF + assert str(UUID(active_lease_list[0])) == active_lease_list[0] def test_leasing_v1_lease_renew(): - response = client.put(f'/leasing/v1/lease/{LEASE_REF}', headers={'authorization': __bearer_token(ORIGIN_REF)}) + response = client.get('/leasing/v1/lessor/leases', headers={'authorization': __bearer_token(ORIGIN_REF)}) + active_lease_list = response.json().get('active_lease_list') + lease_ref = active_lease_list[0] + + ### + + response = client.put(f'/leasing/v1/lease/{lease_ref}', headers={'authorization': __bearer_token(ORIGIN_REF)}) assert response.status_code == 200 - assert response.json()['lease_ref'] == LEASE_REF + assert response.json().get('lease_ref') == lease_ref def test_leasing_v1_lease_delete(): - response = client.delete(f'/leasing/v1/lease/{LEASE_REF}', headers={'authorization': __bearer_token(ORIGIN_REF)}) + response = client.get('/leasing/v1/lessor/leases', headers={'authorization': __bearer_token(ORIGIN_REF)}) + active_lease_list = response.json().get('active_lease_list') + lease_ref = active_lease_list[0] + + ### + + response = client.delete(f'/leasing/v1/lease/{lease_ref}', headers={'authorization': __bearer_token(ORIGIN_REF)}) assert response.status_code == 200 - assert response.json()['lease_ref'] == LEASE_REF + assert response.json().get('lease_ref') == lease_ref def test_leasing_v1_lessor_lease_remove(): - test_leasing_v1_lessor() + lease_ref = test_leasing_v1_lessor() response = client.delete('/leasing/v1/lessor/leases', headers={'authorization': __bearer_token(ORIGIN_REF)}) assert response.status_code == 200 released_lease_list = response.json().get('released_lease_list') assert len(released_lease_list) == 1 - assert released_lease_list[0] == LEASE_REF + assert released_lease_list[0] == lease_ref From 5a5ad0e6542b5b8605efbe6fe2f48c20a4df5732 Mon Sep 17 00:00:00 2001 From: Oscar Krause Date: Tue, 3 Jan 2023 14:09:19 +0100 Subject: [PATCH 16/47] removed 'scope_ref' from code checks because we only support one 'ALLOTMENT_REF', so we need no checks --- app/main.py | 6 +++--- app/orm.py | 22 +++++++++++----------- 2 files changed, 14 insertions(+), 14 deletions(-) diff --git a/app/main.py b/app/main.py index 9259843..303f81d 100644 --- a/app/main.py +++ b/app/main.py @@ -363,8 +363,8 @@ async def leasing_v1_lessor(request: Request): lease_result_list = [] for scope_ref in scope_ref_list: - if scope_ref not in [ALLOTMENT_REF]: - raise HTTPException(status_code=500, detail=f'no service instances found for scopes: ["{scope_ref}"]') + # if scope_ref not in [ALLOTMENT_REF]: + # raise HTTPException(status_code=500, detail=f'no service instances found for scopes: ["{scope_ref}"]') lease_ref = str(uuid4()) expires = cur_time + LEASE_EXPIRE_DELTA @@ -381,7 +381,7 @@ async def leasing_v1_lessor(request: Request): } }) - data = Lease(origin_ref=origin_ref, scope_ref=scope_ref, lease_ref=lease_ref, lease_created=cur_time, lease_expires=expires) + data = Lease(origin_ref=origin_ref, lease_ref=lease_ref, lease_created=cur_time, lease_expires=expires) Lease.create_or_update(db, data) response = { diff --git a/app/orm.py b/app/orm.py index 8128804..1dc5de8 100644 --- a/app/orm.py +++ b/app/orm.py @@ -72,7 +72,7 @@ class Lease(Base): lease_ref = Column(CHAR(length=36), primary_key=True, nullable=False, index=True) # uuid4 origin_ref = Column(CHAR(length=36), ForeignKey(Origin.origin_ref, ondelete='CASCADE'), nullable=False, index=True) # uuid4 - scope_ref = Column(CHAR(length=36), nullable=False, index=True) # uuid4 + # scope_ref = Column(CHAR(length=36), nullable=False, index=True) # uuid4 # not necessary, we only support one scope_ref lease_created = Column(DATETIME(), nullable=False) lease_expires = Column(DATETIME(), nullable=False) lease_updated = Column(DATETIME(), nullable=False) @@ -84,7 +84,7 @@ class Lease(Base): return { 'lease_ref': self.lease_ref, 'origin_ref': self.origin_ref, - 'scope_ref': self.scope_ref, + # 'scope_ref': self.scope_ref, 'lease_created': self.lease_created.isoformat(), 'lease_expires': self.lease_expires.isoformat(), 'lease_updated': self.lease_updated.isoformat(), @@ -180,14 +180,14 @@ def migrate(engine: Engine): Lease.__table__.drop(bind=engine) init(engine) - def upgrade_1_2_to_1_3(): - x = db.dialect.get_columns(engine.connect(), Lease.__tablename__) - x = next((_ for _ in x if _['name'] == 'scope_ref'), None) - if x is None: - Lease.scope_ref.compile() - column_name = Lease.scope_ref.name - column_type = Lease.scope_ref.type.compile(engine.dialect) - engine.execute(f'ALTER TABLE "{Lease.__tablename__}" ADD COLUMN "{column_name}" {column_type}') + # def upgrade_1_2_to_1_3(): + # x = db.dialect.get_columns(engine.connect(), Lease.__tablename__) + # x = next((_ for _ in x if _['name'] == 'scope_ref'), None) + # if x is None: + # Lease.scope_ref.compile() + # column_name = Lease.scope_ref.name + # column_type = Lease.scope_ref.type.compile(engine.dialect) + # engine.execute(f'ALTER TABLE "{Lease.__tablename__}" ADD COLUMN "{column_name}" {column_type}') upgrade_1_0_to_1_1() - upgrade_1_2_to_1_3() + # upgrade_1_2_to_1_3() From 146ae8b82438cff8051f26e971551c635753809e Mon Sep 17 00:00:00 2001 From: Oscar Krause Date: Tue, 3 Jan 2023 14:09:35 +0100 Subject: [PATCH 17/47] updated docs --- doc/Reverse Engineering Notes.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/doc/Reverse Engineering Notes.md b/doc/Reverse Engineering Notes.md index 6746cf7..920a2c3 100644 --- a/doc/Reverse Engineering Notes.md +++ b/doc/Reverse Engineering Notes.md @@ -33,6 +33,8 @@ nvidia-gridd[2986]: License acquired successfully. (Info: license.nvidia.space, Most variables and configs are stored in `/var/lib/docker/volumes/configurations/_data`. +Files can be modified with `docker cp :/venv/... /opt/localfile/...` and back. + ## Dive / Docker image inspector - `dive dls:appliance` From ef1730f4fe1a0fee9661b32721c4c8a5fb68517d Mon Sep 17 00:00:00 2001 From: Oscar Krause Date: Tue, 3 Jan 2023 14:20:13 +0100 Subject: [PATCH 18/47] orm.py - added some docs --- app/orm.py | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/app/orm.py b/app/orm.py index 1dc5de8..1232460 100644 --- a/app/orm.py +++ b/app/orm.py @@ -13,6 +13,7 @@ class Origin(Base): origin_ref = Column(CHAR(length=36), primary_key=True, unique=True, index=True) # uuid4 + # service_instance_xid = Column(CHAR(length=36), nullable=False, index=True) # uuid4 # not necessary, we only support one service_instance_xid ('INSTANCE_REF') hostname = Column(VARCHAR(length=256), nullable=True) guest_driver_version = Column(VARCHAR(length=10), nullable=True) os_platform = Column(VARCHAR(length=256), nullable=True) @@ -24,6 +25,7 @@ class Origin(Base): def serialize(self) -> dict: return { 'origin_ref': self.origin_ref, + # 'service_instance_xid': self.service_instance_xid, 'hostname': self.hostname, 'guest_driver_version': self.guest_driver_version, 'os_platform': self.os_platform, @@ -72,7 +74,7 @@ class Lease(Base): lease_ref = Column(CHAR(length=36), primary_key=True, nullable=False, index=True) # uuid4 origin_ref = Column(CHAR(length=36), ForeignKey(Origin.origin_ref, ondelete='CASCADE'), nullable=False, index=True) # uuid4 - # scope_ref = Column(CHAR(length=36), nullable=False, index=True) # uuid4 # not necessary, we only support one scope_ref + # scope_ref = Column(CHAR(length=36), nullable=False, index=True) # uuid4 # not necessary, we only support one scope_ref ('ALLOTMENT_REF') lease_created = Column(DATETIME(), nullable=False) lease_expires = Column(DATETIME(), nullable=False) lease_updated = Column(DATETIME(), nullable=False) From b64c5318985d6ad010bb45ca47116011ef4858b8 Mon Sep 17 00:00:00 2001 From: Oscar Krause Date: Tue, 3 Jan 2023 14:50:52 +0100 Subject: [PATCH 19/47] bump version to 1.3 --- version.env | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/version.env b/version.env index 955cc3f..5be527b 100644 --- a/version.env +++ b/version.env @@ -1 +1 @@ -VERSION=1.2 +VERSION=1.3 From 18e9ab2ebffbf140c38da36ee93959138b859f3a Mon Sep 17 00:00:00 2001 From: Oscar Krause Date: Tue, 3 Jan 2023 14:52:31 +0100 Subject: [PATCH 20/47] fixes --- app/orm.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/app/orm.py b/app/orm.py index 1232460..efc5853 100644 --- a/app/orm.py +++ b/app/orm.py @@ -80,7 +80,7 @@ class Lease(Base): lease_updated = Column(DATETIME(), nullable=False) def __repr__(self): - return f'Lease(origin_ref={self.origin_ref}, scope_ref={self.scope_ref}, lease_ref={self.lease_ref}, expires={self.lease_expires})' + return f'Lease(origin_ref={self.origin_ref}, lease_ref={self.lease_ref}, expires={self.lease_expires})' def serialize(self) -> dict: return { From 01fe142850550b2657805f77abbe0d5ee76d2c03 Mon Sep 17 00:00:00 2001 From: Oscar Krause Date: Tue, 3 Jan 2023 15:22:49 +0100 Subject: [PATCH 21/47] .gitlab-ci.yml - fixed release job --- .gitlab-ci.yml | 25 ++++++++++++++++++------- 1 file changed, 18 insertions(+), 7 deletions(-) diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 0a0d788..6f0e7a8 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -267,20 +267,31 @@ deploy:pacman: - 'echo "EXPORT_NAME: ${EXPORT_NAME}"' - 'curl --header "JOB-TOKEN: $CI_JOB_TOKEN" --upload-file ${EXPORT_NAME} "${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/packages/generic/${PACKAGE_NAME}/${PACKAGE_VERSION}/${EXPORT_NAME}"' -release: - image: registry.gitlab.com/gitlab-org/release-cli:latest - stage: .post +release:prepare: + stage: .pre rules: - if: $CI_COMMIT_TAG when: never - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH - before_script: - - set -a # make variables from "source" command available to release-cli + script: - source version.env + - echo &VERSION + artifacts: + reports: + dotenv: version.env + +release: + image: registry.gitlab.com/gitlab-org/release-cli:latest + stage: .post + needs: + - job: release:prepare + artifacts: true + rules: + - if: $CI_COMMIT_TAG + when: never + - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH script: - echo "Running release-job for $VERSION" - after_script: - - set +a release: name: $CI_PROJECT_TITLE $version description: Release of $CI_PROJECT_TITLE version $VERSION From d6cc6dcbee2773997160dae602de596e96fa5af9 Mon Sep 17 00:00:00 2001 From: Oscar Krause Date: Tue, 3 Jan 2023 17:38:32 +0100 Subject: [PATCH 22/47] fixes --- app/main.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/app/main.py b/app/main.py index 303f81d..6012e4d 100644 --- a/app/main.py +++ b/app/main.py @@ -491,7 +491,7 @@ async def leasing_v1_lessor_lease_remove(request: Request): @app.post('/leasing/v1/lessor/shutdown', description='shutdown all leases') async def leasing_v1_lessor_shutdown(request: Request): - j, cur_time = json.loads((await request.body()).decode('utf-8')) + j, cur_time = json.loads((await request.body()).decode('utf-8')), datetime.utcnow() token = j.get('token') token = jwt.decode(token=token, key=jwt_decode_key, algorithms=ALGORITHMS.RS256, options={'verify_aud': False}) From f1369d5e258324ebb920ff7b6c1fae2d96321e20 Mon Sep 17 00:00:00 2001 From: Oscar Krause Date: Tue, 3 Jan 2023 17:38:45 +0100 Subject: [PATCH 23/47] added some docs --- doc/Reverse Engineering Notes.md | 1 + 1 file changed, 1 insertion(+) diff --git a/doc/Reverse Engineering Notes.md b/doc/Reverse Engineering Notes.md index 920a2c3..0233a64 100644 --- a/doc/Reverse Engineering Notes.md +++ b/doc/Reverse Engineering Notes.md @@ -34,6 +34,7 @@ nvidia-gridd[2986]: License acquired successfully. (Info: license.nvidia.space, Most variables and configs are stored in `/var/lib/docker/volumes/configurations/_data`. Files can be modified with `docker cp :/venv/... /opt/localfile/...` and back. +(May you need to fix permissions with `docker exec -u 0 chown nonroot:nonroot /venv/...`) ## Dive / Docker image inspector From 5575fee382cfea6b85ee15c429f36ed681904d08 Mon Sep 17 00:00:00 2001 From: Oscar Krause Date: Tue, 3 Jan 2023 17:39:10 +0100 Subject: [PATCH 24/47] fixed config test --- test/main.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/test/main.py b/test/main.py index f04de99..48a99bf 100644 --- a/test/main.py +++ b/test/main.py @@ -50,7 +50,7 @@ def test_health(): def test_config(): - response = client.get('/-/') + response = client.get('/-/config') assert response.status_code == 200 From ac811d5df79171ba63fa9a82b818ca4ace5c55a0 Mon Sep 17 00:00:00 2001 From: Oscar Krause Date: Tue, 3 Jan 2023 18:04:21 +0100 Subject: [PATCH 25/47] added 'LEASE_EXPIRE_HOURS' variable for better debugging --- app/main.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/app/main.py b/app/main.py index 6012e4d..7569522 100644 --- a/app/main.py +++ b/app/main.py @@ -41,7 +41,7 @@ ALLOTMENT_REF = str(env('ALLOTMENT_REF', '20000000-0000-0000-0000-000000000001') INSTANCE_KEY_RSA = load_key(str(env('INSTANCE_KEY_RSA', join(dirname(__file__), 'cert/instance.private.pem')))) INSTANCE_KEY_PUB = load_key(str(env('INSTANCE_KEY_PUB', join(dirname(__file__), 'cert/instance.public.pem')))) TOKEN_EXPIRE_DELTA = relativedelta(hours=1) # days=1 -LEASE_EXPIRE_DELTA = relativedelta(days=int(env('LEASE_EXPIRE_DAYS', 90))) +LEASE_EXPIRE_DELTA = relativedelta(days=int(env('LEASE_EXPIRE_DAYS', 90)), hours=int(env('LEASE_EXPIRE_HOURS', 0))) LEASE_RENEWAL_PERIOD = float(env('LEASE_RENEWAL_PERIOD', 0.15)) CORS_ORIGINS = str(env('CORS_ORIGINS', '')).split(',') if (env('CORS_ORIGINS')) else [f'https://{DLS_URL}'] From 7b2428ea386207628a14c6821c6f939544f41858 Mon Sep 17 00:00:00 2001 From: Oscar Krause Date: Tue, 3 Jan 2023 18:04:50 +0100 Subject: [PATCH 26/47] removed some debugging --- app/orm.py | 1 - 1 file changed, 1 deletion(-) diff --git a/app/orm.py b/app/orm.py index efc5853..aadc0ef 100644 --- a/app/orm.py +++ b/app/orm.py @@ -41,7 +41,6 @@ class Origin(Base): def create_or_update(engine: Engine, origin: "Origin"): session = sessionmaker(bind=engine)() entity = session.query(Origin).filter(Origin.origin_ref == origin.origin_ref).first() - print(entity) if entity is None: session.add(origin) else: From 7e6e523799067dbc666efc251380fa51c692ea35 Mon Sep 17 00:00:00 2001 From: Oscar Krause Date: Tue, 3 Jan 2023 18:05:12 +0100 Subject: [PATCH 27/47] improved test (checking uuid are 36 chars long) --- test/main.py | 20 ++++++++++++++------ 1 file changed, 14 insertions(+), 6 deletions(-) diff --git a/test/main.py b/test/main.py index 48a99bf..67856c9 100644 --- a/test/main.py +++ b/test/main.py @@ -184,7 +184,9 @@ def test_leasing_v1_lessor(): lease_result_list = response.json().get('lease_result_list') assert len(lease_result_list) == 1 + assert len(lease_result_list[0]['lease']['ref']) == 36 assert str(UUID(lease_result_list[0]['lease']['ref'])) == lease_result_list[0]['lease']['ref'] + return lease_result_list[0]['lease']['ref'] @@ -194,33 +196,38 @@ def test_leasing_v1_lessor_lease(): active_lease_list = response.json().get('active_lease_list') assert len(active_lease_list) == 1 + assert len(active_lease_list[0]) == 36 assert str(UUID(active_lease_list[0])) == active_lease_list[0] def test_leasing_v1_lease_renew(): response = client.get('/leasing/v1/lessor/leases', headers={'authorization': __bearer_token(ORIGIN_REF)}) active_lease_list = response.json().get('active_lease_list') - lease_ref = active_lease_list[0] + active_lease_ref = active_lease_list[0] ### - response = client.put(f'/leasing/v1/lease/{lease_ref}', headers={'authorization': __bearer_token(ORIGIN_REF)}) + response = client.put(f'/leasing/v1/lease/{active_lease_ref}', headers={'authorization': __bearer_token(ORIGIN_REF)}) assert response.status_code == 200 - assert response.json().get('lease_ref') == lease_ref + lease_ref = response.json().get('lease_ref') + assert len(lease_ref) == 36 + assert lease_ref == active_lease_ref def test_leasing_v1_lease_delete(): response = client.get('/leasing/v1/lessor/leases', headers={'authorization': __bearer_token(ORIGIN_REF)}) active_lease_list = response.json().get('active_lease_list') - lease_ref = active_lease_list[0] + active_lease_ref = active_lease_list[0] ### - response = client.delete(f'/leasing/v1/lease/{lease_ref}', headers={'authorization': __bearer_token(ORIGIN_REF)}) + response = client.delete(f'/leasing/v1/lease/{active_lease_ref}', headers={'authorization': __bearer_token(ORIGIN_REF)}) assert response.status_code == 200 - assert response.json().get('lease_ref') == lease_ref + lease_ref = response.json().get('lease_ref') + assert len(lease_ref) == 36 + assert lease_ref == active_lease_ref def test_leasing_v1_lessor_lease_remove(): @@ -231,4 +238,5 @@ def test_leasing_v1_lessor_lease_remove(): released_lease_list = response.json().get('released_lease_list') assert len(released_lease_list) == 1 + assert len(released_lease_list[0]) == 36 assert released_lease_list[0] == lease_ref From 41980212129c4bffcfce9297d5bc768c66b98b7a Mon Sep 17 00:00:00 2001 From: Oscar Krause Date: Tue, 3 Jan 2023 18:10:02 +0100 Subject: [PATCH 28/47] README.md - fixed windows issue with `/leasing/v1/lessor/shutdown` --- README.md | 25 ++++++++++- docker-compose.yml | 108 +++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 131 insertions(+), 2 deletions(-) create mode 100644 docker-compose.yml diff --git a/README.md b/README.md index e64effa..9642367 100644 --- a/README.md +++ b/README.md @@ -11,7 +11,8 @@ Only the clients need a connection to this service on configured port. ## ToDo's -- Support http mode for using external https proxy (disable uvicorn ssl for using behind proxy) +- check why windows guests display "can't acquire license" although in log there is no message displayed and license is + also acquired successfully ## Endpoints @@ -102,6 +103,8 @@ docker run -e DLS_URL=`hostname -i` -e DLS_PORT=443 -p 443:443 -v $WORKING_DIR:/ **Docker-Compose / Deploy stack** +Goto [`docker-compose.yml`](docker-compose.yml) for more advanced example. + ```yaml version: '3.9' @@ -439,7 +442,10 @@ Dec 20 17:53:34 ubuntu-grid-server nvidia-gridd[10354]: License acquired success -### Error on releasing leases on shutdown +### Error on releasing leases on shutdown (fixed in 1.3 by using reverse proxy) + +**UPDATE for version `1.3`**: This issue can be fixed by using a reverse proxy (e.g. `nginx`). Please read section +below. The driver wants to release current leases on shutting down windows. This endpoint needs to be a http endpoint and is currently not implemented. The error message looks like and safely can be ignored (since we have no license @@ -452,6 +458,21 @@ limitation :P): <0>:End Logging ``` +#### log with 1.3 and nginx as reverse proxy + +``` +<1>:NLS initialized +<2>:NLS initialized +<1>:Valid GRID license not found. GPU features and performance will be fully degraded. To enable full functionality please configure licensing details. +<1>:License acquired successfully. (Info: 192.168.178.33, NVIDIA RTX Virtual Workstation; Expiry: 2023-1-4 16:48:20 GMT) +<2>:Valid GRID license not found. GPU features and performance will be fully degraded. To enable full functionality please configure licensing details. +<2>:License acquired successfully from local trusted store. (Info: 192.168.178.33, NVIDIA RTX Virtual Workstation; Expiry: 2023-1-4 16:48:20 GMT) +<2>:End Logging +<1>:End Logging +<0>:License returned successfully. (Info: 192.168.178.33) +<0>:End Logging +``` + # Credits Thanks to vGPU community and all who uses this project and report bugs. diff --git a/docker-compose.yml b/docker-compose.yml new file mode 100644 index 0000000..2411067 --- /dev/null +++ b/docker-compose.yml @@ -0,0 +1,108 @@ +version: '3.9' + +x-dls-variables: &dls-variables + DLS_URL: localhost # REQUIRED + DLS_PORT: 443 # must match nginx listen port + LEASE_EXPIRE_DAYS: 90 + DATABASE: sqlite:////app/database/db.sqlite + DEBUG: false + +services: + web: + image: nginx + ports: + # thees are ports where nginx (!) is listen to + - "80:80" # for "/leasing/v1/lessor/shutdown" used by windows guests, can't be changed! + - "443:443" # first part must match "DLS_PORT" + volumes: + - /opt/docker/fastapi-dls/cert:/opt/cert + healthcheck: + test: [ "CMD", "curl", "--insecure", "--fail", "https://localhost/-/health" ] + interval: 10s + timeout: 5s + retries: 3 + start_period: 30s + command: | + bash -c 'bash -s <<"EOF" + cat > /etc/nginx/nginx.conf <<"EON" + daemon off; + user root; + worker_processes auto; + + events { + worker_connections 1024; + } + + http { + gzip on; + gzip_disable "msie6"; + include /etc/nginx/mime.types; + + upstream dls-backend { + server dls:443; + } + + server { + listen 443 ssl http2 default_server; + listen [::]:443 ssl http2 default_server; + + root /var/www/html; + index index.html; + server_name _; + + ssl_certificate "/opt/cert/webserver.crt"; + ssl_certificate_key "/opt/cert/webserver.key"; + ssl_session_cache shared:SSL:1m; + ssl_session_timeout 10m; + ssl_protocols TLSv1.3 TLSv1.2; + # ssl_ciphers "ECDHE-ECDSA-CHACHA20-POLY1305"; + # ssl_ciphers PROFILE=SYSTEM; + ssl_prefer_server_ciphers on; + + location / { + proxy_ssl_verify off; + proxy_set_header Host $$http_host; + proxy_set_header X-Real-IP $$remote_addr; + proxy_set_header X-Forwarded-For $$proxy_add_x_forwarded_for; + proxy_set_header X-Forwarded-Proto $$scheme; + proxy_pass https://dls-backend$$request_uri; + } + } + + server { + listen 80; + listen [::]:80; + + root /var/www/html; + index index.html; + server_name _; + + location /leasing/v1/lessor/shutdown { + proxy_ssl_verify off; + proxy_set_header Host $$http_host; + proxy_set_header X-Real-IP $$remote_addr; + proxy_set_header X-Forwarded-For $$proxy_add_x_forwarded_for; + proxy_set_header X-Forwarded-Proto $$scheme; + proxy_pass https://dls-backend/leasing/v1/lessor/shutdown; + } + + location / { + return 301 https://dls-backend$$request_uri; + } + } + } + EON + cat /etc/nginx/nginx.conf + nginx + EOF' + dls: + image: collinwebdesigns/fastapi-dls:latest + restart: always + environment: + <<: *dls-variables + volumes: + - /opt/docker/fastapi-dls/cert:/app/cert + - db:/app/database + +volumes: + db: From 7e3f2d0345296179a367ca204a89e0a5c087381e Mon Sep 17 00:00:00 2001 From: Oscar Krause Date: Tue, 3 Jan 2023 18:44:30 +0100 Subject: [PATCH 29/47] docker-compose.yml - fixes --- docker-compose.yml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docker-compose.yml b/docker-compose.yml index 2411067..bb33609 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -87,14 +87,14 @@ services: } location / { - return 301 https://dls-backend$$request_uri; + return 301 https://$$host$$request_uri; } } } EON - cat /etc/nginx/nginx.conf nginx EOF' + dls: image: collinwebdesigns/fastapi-dls:latest restart: always From 5bb8437b1d615febc94cc119f4abef9dc7507908 Mon Sep 17 00:00:00 2001 From: Oscar Krause Date: Tue, 3 Jan 2023 18:59:34 +0100 Subject: [PATCH 30/47] README.md - added timestamp to linux token filename --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 9642367..4e96236 100644 --- a/README.md +++ b/README.md @@ -319,7 +319,7 @@ Successfully tested with this package versions: ## Linux ```shell -curl --insecure -L -X GET https:///client-token -o /etc/nvidia/ClientConfigToken/client_configuration_token.tok +curl --insecure -L -X GET https:///client-token -o /etc/nvidia/ClientConfigToken/client_configuration_token_$(date '+%d-%m-%Y-%H-%M-%S').tok service nvidia-gridd restart nvidia-smi -q | grep "License" ``` From 27f47b93b88f1fc88ef4896ac1f74b05638d3c75 Mon Sep 17 00:00:00 2001 From: Oscar Krause Date: Tue, 3 Jan 2023 20:45:16 +0100 Subject: [PATCH 31/47] docker-compose.yml - added experimental health endpoint --- docker-compose.yml | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/docker-compose.yml b/docker-compose.yml index bb33609..77dcc07 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -67,6 +67,12 @@ services: proxy_set_header X-Forwarded-Proto $$scheme; proxy_pass https://dls-backend$$request_uri; } + + location = /-/health { + access_log off; + add_header 'Content-Type' 'application/json'; + return 200; # '{\"status\":\"up\",\"service\":\"nginx\"}'; + } } server { From 8bd37c0ead6975f503300c6c1597330be1769e48 Mon Sep 17 00:00:00 2001 From: Oscar Krause Date: Wed, 4 Jan 2023 07:40:37 +0100 Subject: [PATCH 32/47] added some notes to required variables to change --- README.md | 2 +- docker-compose.yml | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 4e96236..ed3aa43 100644 --- a/README.md +++ b/README.md @@ -109,7 +109,7 @@ Goto [`docker-compose.yml`](docker-compose.yml) for more advanced example. version: '3.9' x-dls-variables: &dls-variables - DLS_URL: localhost # REQUIRED + DLS_URL: localhost # REQUIRED, change to your ip or hostname DLS_PORT: 443 LEASE_EXPIRE_DAYS: 90 DATABASE: sqlite:////app/database/db.sqlite diff --git a/docker-compose.yml b/docker-compose.yml index 77dcc07..2ebd525 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -1,7 +1,7 @@ version: '3.9' x-dls-variables: &dls-variables - DLS_URL: localhost # REQUIRED + DLS_URL: localhost # REQUIRED, change to your ip or hostname DLS_PORT: 443 # must match nginx listen port LEASE_EXPIRE_DAYS: 90 DATABASE: sqlite:////app/database/db.sqlite From b839e6c2b33085f494a88d09c201e977b96cb467 Mon Sep 17 00:00:00 2001 From: Oscar Krause Date: Wed, 4 Jan 2023 10:04:52 +0100 Subject: [PATCH 33/47] code styling - replaced 'json.loads' with 'json_loads' - shortened 'JSONResponse' to 'JSONr' - shortened 'HTMLResponse' to 'HTMLr' - replaced HTTPException with JsonResponses - added some error handing for invalid tokens --- app/main.py | 82 ++++++++++++++++++++++++++++++----------------------- 1 file changed, 46 insertions(+), 36 deletions(-) diff --git a/app/main.py b/app/main.py index 7569522..d570b8e 100644 --- a/app/main.py +++ b/app/main.py @@ -6,16 +6,16 @@ from os.path import join, dirname from os import getenv as env from dotenv import load_dotenv -from fastapi import FastAPI, HTTPException +from fastapi import FastAPI from fastapi.requests import Request -import json +from json import loads as json_loads from datetime import datetime from dateutil.relativedelta import relativedelta from calendar import timegm -from jose import jws, jwk, jwt +from jose import jws, jwk, jwt, JWTError from jose.constants import ALGORITHMS from starlette.middleware.cors import CORSMiddleware -from starlette.responses import StreamingResponse, JSONResponse, HTMLResponse, Response, RedirectResponse +from starlette.responses import StreamingResponse, JSONResponse as JSONr, HTMLResponse as HTMLr, Response, RedirectResponse from sqlalchemy import create_engine from sqlalchemy.orm import sessionmaker @@ -78,12 +78,12 @@ async def _index(): @app.get('/-/health', summary='* Health') async def _health(request: Request): - return JSONResponse({'status': 'up'}) + return JSONr({'status': 'up'}) @app.get('/-/config', summary='* Config', description='returns environment variables.') async def _config(): - return JSONResponse({ + return JSONr({ 'VERSION': str(VERSION), 'COMMIT': str(COMMIT), 'DEBUG': str(DEBUG), @@ -91,7 +91,7 @@ async def _config(): 'DLS_PORT': str(DLS_PORT), 'SITE_KEY_XID': str(SITE_KEY_XID), 'INSTANCE_REF': str(INSTANCE_REF), - 'ALLOTMENT_REF': [ALLOTMENT_REF], + 'ALLOTMENT_REF': [str(ALLOTMENT_REF)], 'TOKEN_EXPIRE_DELTA': str(TOKEN_EXPIRE_DELTA), 'LEASE_EXPIRE_DELTA': str(LEASE_EXPIRE_DELTA), 'LEASE_RENEWAL_PERIOD': str(LEASE_RENEWAL_PERIOD), @@ -103,7 +103,7 @@ async def _config(): async def _readme(): from markdown import markdown content = load_file('../README.md').decode('utf-8') - return HTMLResponse(markdown(text=content, extensions=['tables', 'fenced_code', 'md_in_html', 'nl2br', 'toc'])) + return HTMLr(markdown(text=content, extensions=['tables', 'fenced_code', 'md_in_html', 'nl2br', 'toc'])) @app.get('/-/manage', summary='* Management UI') @@ -137,7 +137,7 @@ async def _manage(request: Request): ''' - return HTMLResponse(response) + return HTMLr(response) @app.get('/-/origins', summary='* Origins') @@ -150,7 +150,7 @@ async def _origins(request: Request, leases: bool = False): x['leases'] = list(map(lambda _: _.serialize(), Lease.find_by_origin_ref(db, origin.origin_ref))) response.append(x) session.close() - return JSONResponse(response) + return JSONr(response) @app.delete('/-/origins', summary='* Origins') @@ -170,14 +170,14 @@ async def _leases(request: Request, origin: bool = False): x['origin'] = session.query(Origin).filter(Origin.origin_ref == lease.origin_ref).first().serialize() response.append(x) session.close() - return JSONResponse(response) + return JSONr(response) @app.delete('/-/lease/{lease_ref}', summary='* Lease') async def _lease_delete(request: Request, lease_ref: str): if Lease.delete(db, lease_ref) == 1: return Response(status_code=201) - raise HTTPException(status_code=404, detail='lease not found') + return JSONr(status_code=404, content={'status': 404, 'detail': 'lease not found'}) # venv/lib/python3.9/site-packages/nls_core_service_instance/service_instance_token_manager.py @@ -229,7 +229,7 @@ async def _client_token(): # venv/lib/python3.9/site-packages/nls_services_auth/test/test_origins_controller.py @app.post('/auth/v1/origin', description='find or create an origin') async def auth_v1_origin(request: Request): - j, cur_time = json.loads((await request.body()).decode('utf-8')), datetime.utcnow() + j, cur_time = json_loads((await request.body()).decode('utf-8')), datetime.utcnow() origin_ref = j.get('candidate_origin_ref') logging.info(f'> [ origin ]: {origin_ref}: {j}') @@ -253,13 +253,13 @@ async def auth_v1_origin(request: Request): "sync_timestamp": cur_time.isoformat() } - return JSONResponse(response) + return JSONr(response) # venv/lib/python3.9/site-packages/nls_services_auth/test/test_origins_controller.py @app.post('/auth/v1/origin/update', description='update an origin evidence') async def auth_v1_origin_update(request: Request): - j, cur_time = json.loads((await request.body()).decode('utf-8')), datetime.utcnow() + j, cur_time = json_loads((await request.body()).decode('utf-8')), datetime.utcnow() origin_ref = j.get('origin_ref') logging.info(f'> [ update ]: {origin_ref}: {j}') @@ -279,14 +279,14 @@ async def auth_v1_origin_update(request: Request): "sync_timestamp": cur_time.isoformat() } - return JSONResponse(response) + return JSONr(response) # venv/lib/python3.9/site-packages/nls_services_auth/test/test_auth_controller.py # venv/lib/python3.9/site-packages/nls_core_auth/auth.py - CodeResponse @app.post('/auth/v1/code', description='get an authorization code') async def auth_v1_code(request: Request): - j, cur_time = json.loads((await request.body()).decode('utf-8')), datetime.utcnow() + j, cur_time = json_loads((await request.body()).decode('utf-8')), datetime.utcnow() origin_ref = j.get('origin_ref') logging.info(f'> [ code ]: {origin_ref}: {j}') @@ -311,22 +311,27 @@ async def auth_v1_code(request: Request): "prompts": None } - return JSONResponse(response) + return JSONr(response) # venv/lib/python3.9/site-packages/nls_services_auth/test/test_auth_controller.py # venv/lib/python3.9/site-packages/nls_core_auth/auth.py - TokenResponse @app.post('/auth/v1/token', description='exchange auth code and verifier for token') async def auth_v1_token(request: Request): - j, cur_time = json.loads((await request.body()).decode('utf-8')), datetime.utcnow() - payload = jwt.decode(token=j.get('auth_code'), key=jwt_decode_key) + j, cur_time = json_loads((await request.body()).decode('utf-8')), datetime.utcnow() + + try: + payload = jwt.decode(token=j.get('auth_code'), key=jwt_decode_key) + except JWTError as e: + return JSONr(status_code=400, content={'status': 400, 'title': 'invalid token', 'detail': str(e)}) origin_ref = payload.get('origin_ref') logging.info(f'> [ auth ]: {origin_ref}: {j}') # validate the code challenge - if payload.get('challenge') != b64enc(sha256(j.get('code_verifier').encode('utf-8')).digest()).rstrip(b'=').decode('utf-8'): - raise HTTPException(status_code=401, detail='expected challenge did not match verifier') + challenge = b64enc(sha256(j.get('code_verifier').encode('utf-8')).digest()).rstrip(b'=').decode('utf-8') + if payload.get('challenge') != challenge: + raise JSONr(status_code=401, content={'status': 401, 'detail': 'expected challenge did not match verifier'}) access_expires_on = cur_time + TOKEN_EXPIRE_DELTA @@ -349,13 +354,18 @@ async def auth_v1_token(request: Request): "sync_timestamp": cur_time.isoformat(), } - return JSONResponse(response) + return JSONr(response) # venv/lib/python3.9/site-packages/nls_services_lease/test/test_lease_multi_controller.py @app.post('/leasing/v1/lessor', description='request multiple leases (borrow) for current origin') async def leasing_v1_lessor(request: Request): - j, token, cur_time = json.loads((await request.body()).decode('utf-8')), __get_token(request), datetime.utcnow() + j, token, cur_time = json_loads((await request.body()).decode('utf-8')), __get_token(request), datetime.utcnow() + + try: + token = __get_token(request) + except JWTError: + return JSONr(status_code=401, content={'status': 401, 'detail': 'token is not valid'}) origin_ref = token.get('origin_ref') scope_ref_list = j.get('scope_ref_list') @@ -364,7 +374,7 @@ async def leasing_v1_lessor(request: Request): lease_result_list = [] for scope_ref in scope_ref_list: # if scope_ref not in [ALLOTMENT_REF]: - # raise HTTPException(status_code=500, detail=f'no service instances found for scopes: ["{scope_ref}"]') + # raise JSONr(status_code=500, detail=f'no service instances found for scopes: ["{scope_ref}"]') lease_ref = str(uuid4()) expires = cur_time + LEASE_EXPIRE_DELTA @@ -391,7 +401,7 @@ async def leasing_v1_lessor(request: Request): "prompts": None } - return JSONResponse(response) + return JSONr(response) # venv/lib/python3.9/site-packages/nls_services_lease/test/test_lease_multi_controller.py @@ -411,7 +421,7 @@ async def leasing_v1_lessor_lease(request: Request): "prompts": None } - return JSONResponse(response) + return JSONr(response) # venv/lib/python3.9/site-packages/nls_services_lease/test/test_lease_single_controller.py @@ -425,7 +435,7 @@ async def leasing_v1_lease_renew(request: Request, lease_ref: str): entity = Lease.find_by_origin_ref_and_lease_ref(db, origin_ref, lease_ref) if entity is None: - raise HTTPException(status_code=404, detail='requested lease not available') + return JSONr(status_code=404, content={'status': 404, 'detail': 'requested lease not available'}) expires = cur_time + LEASE_EXPIRE_DELTA response = { @@ -439,7 +449,7 @@ async def leasing_v1_lease_renew(request: Request, lease_ref: str): Lease.renew(db, entity, expires, cur_time) - return JSONResponse(response) + return JSONr(response) # venv/lib/python3.9/site-packages/nls_services_lease/test/test_lease_single_controller.py @@ -452,12 +462,12 @@ async def leasing_v1_lease_delete(request: Request, lease_ref: str): entity = Lease.find_by_lease_ref(db, lease_ref) if entity.origin_ref != origin_ref: - raise HTTPException(status_code=403, detail='access or operation forbidden') + return JSONr(status_code=403, content={'status': 403, 'detail': 'access or operation forbidden'}) if entity is None: - raise HTTPException(status_code=404, detail='requested lease not available') + return JSONr(status_code=404, content={'status': 404, 'detail': 'requested lease not available'}) if Lease.delete(db, lease_ref) == 0: - raise HTTPException(status_code=404, detail='lease not found') + return JSONr(status_code=404, content={'status': 404, 'detail': 'lease not found'}) response = { "lease_ref": lease_ref, @@ -465,7 +475,7 @@ async def leasing_v1_lease_delete(request: Request, lease_ref: str): "sync_timestamp": cur_time.isoformat(), } - return JSONResponse(response) + return JSONr(response) # venv/lib/python3.9/site-packages/nls_services_lease/test/test_lease_multi_controller.py @@ -486,12 +496,12 @@ async def leasing_v1_lessor_lease_remove(request: Request): "prompts": None } - return JSONResponse(response) + return JSONr(response) @app.post('/leasing/v1/lessor/shutdown', description='shutdown all leases') async def leasing_v1_lessor_shutdown(request: Request): - j, cur_time = json.loads((await request.body()).decode('utf-8')), datetime.utcnow() + j, cur_time = json_loads((await request.body()).decode('utf-8')), datetime.utcnow() token = j.get('token') token = jwt.decode(token=token, key=jwt_decode_key, algorithms=ALGORITHMS.RS256, options={'verify_aud': False}) @@ -508,7 +518,7 @@ async def leasing_v1_lessor_shutdown(request: Request): "prompts": None } - return JSONResponse(response) + return JSONr(response) if __name__ == '__main__': From 6c9ea63dc14bc141e26f6d841ff5e95b911c29f3 Mon Sep 17 00:00:00 2001 From: Oscar Krause Date: Wed, 4 Jan 2023 10:08:17 +0100 Subject: [PATCH 34/47] added variable for TOKEN_EXPIRE_DELTA --- README.md | 1 + app/main.py | 2 +- 2 files changed, 2 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index ed3aa43..e97b8df 100644 --- a/README.md +++ b/README.md @@ -290,6 +290,7 @@ After first success you have to replace `--issue` with `--renew`. | `DEBUG` | `false` | Toggles `fastapi` debug mode | | `DLS_URL` | `localhost` | Used in client-token to tell guest driver where dls instance is reachable | | `DLS_PORT` | `443` | Used in client-token to tell guest driver where dls instance is reachable | +| `TOKEN_EXPIRE_DAYS` | `1` | Client auth-token validity (used for authenticate client against api, **not `.tok` file!**) | | `LEASE_EXPIRE_DAYS` | `90` | Lease time in days | | `LEASE_RENEWAL_PERIOD` | `0.15` | The percentage of the lease period that must elapse before a licensed client can renew a license \*1 | | `DATABASE` | `sqlite:///db.sqlite` | See [official SQLAlchemy docs](https://docs.sqlalchemy.org/en/14/core/engines.html) | diff --git a/app/main.py b/app/main.py index d570b8e..d688768 100644 --- a/app/main.py +++ b/app/main.py @@ -40,7 +40,7 @@ INSTANCE_REF = str(env('INSTANCE_REF', '10000000-0000-0000-0000-000000000001')) ALLOTMENT_REF = str(env('ALLOTMENT_REF', '20000000-0000-0000-0000-000000000001')) INSTANCE_KEY_RSA = load_key(str(env('INSTANCE_KEY_RSA', join(dirname(__file__), 'cert/instance.private.pem')))) INSTANCE_KEY_PUB = load_key(str(env('INSTANCE_KEY_PUB', join(dirname(__file__), 'cert/instance.public.pem')))) -TOKEN_EXPIRE_DELTA = relativedelta(hours=1) # days=1 +TOKEN_EXPIRE_DELTA = relativedelta(days=int(env('TOKEN_EXPIRE_DAYS', 1)), hours=int(env('TOKEN_EXPIRE_HOURS', 0))) LEASE_EXPIRE_DELTA = relativedelta(days=int(env('LEASE_EXPIRE_DAYS', 90)), hours=int(env('LEASE_EXPIRE_HOURS', 0))) LEASE_RENEWAL_PERIOD = float(env('LEASE_RENEWAL_PERIOD', 0.15)) CORS_ORIGINS = str(env('CORS_ORIGINS', '')).split(',') if (env('CORS_ORIGINS')) else [f'https://{DLS_URL}'] From 32b05808c451f75575f205478d2a8da26585e9ad Mon Sep 17 00:00:00 2001 From: Oscar Krause Date: Wed, 4 Jan 2023 10:14:00 +0100 Subject: [PATCH 35/47] fixed "return" instead of "raise" --- app/main.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/app/main.py b/app/main.py index d688768..3bb2a3a 100644 --- a/app/main.py +++ b/app/main.py @@ -331,7 +331,7 @@ async def auth_v1_token(request: Request): # validate the code challenge challenge = b64enc(sha256(j.get('code_verifier').encode('utf-8')).digest()).rstrip(b'=').decode('utf-8') if payload.get('challenge') != challenge: - raise JSONr(status_code=401, content={'status': 401, 'detail': 'expected challenge did not match verifier'}) + return JSONr(status_code=401, content={'status': 401, 'detail': 'expected challenge did not match verifier'}) access_expires_on = cur_time + TOKEN_EXPIRE_DELTA @@ -374,7 +374,7 @@ async def leasing_v1_lessor(request: Request): lease_result_list = [] for scope_ref in scope_ref_list: # if scope_ref not in [ALLOTMENT_REF]: - # raise JSONr(status_code=500, detail=f'no service instances found for scopes: ["{scope_ref}"]') + # return JSONr(status_code=500, detail=f'no service instances found for scopes: ["{scope_ref}"]') lease_ref = str(uuid4()) expires = cur_time + LEASE_EXPIRE_DELTA From 5f5569a0c7e3f078a4ecb656d8c317e097784b41 Mon Sep 17 00:00:00 2001 From: Oscar Krause Date: Wed, 4 Jan 2023 11:02:54 +0100 Subject: [PATCH 36/47] improved debian installation --- .DEBIAN/env.default | 27 ++++++++++++ .DEBIAN/fastapi-dls.service | 25 +++++++++++ .DEBIAN/postinst | 87 +++++-------------------------------- .gitlab-ci.yml | 6 ++- 4 files changed, 69 insertions(+), 76 deletions(-) create mode 100644 .DEBIAN/env.default create mode 100644 .DEBIAN/fastapi-dls.service diff --git a/.DEBIAN/env.default b/.DEBIAN/env.default new file mode 100644 index 0000000..835f29e --- /dev/null +++ b/.DEBIAN/env.default @@ -0,0 +1,27 @@ +# Toggle debug mode +#DEBUG=false + +# Where the client can find the DLS server +DLS_URL=127.0.0.1 +DLS_PORT=443 + +# CORS configuration +## comma separated list without spaces +#CORS_ORIGINS="https://$DLS_URL:$DLS_PORT" + +# Lease expiration in days +LEASE_EXPIRE_DAYS=90 +LEASE_RENEWAL_PERIOD=0.2 + +# Database location +## https://docs.sqlalchemy.org/en/14/core/engines.html +DATABASE=sqlite:////etc/fastapi-dls/db.sqlite + +# UUIDs for identifying the instance +#SITE_KEY_XID="00000000-0000-0000-0000-000000000000" +#INSTANCE_REF="10000000-0000-0000-0000-000000000001" +#ALLOTMENT_REF="20000000-0000-0000-0000-000000000001" + +# Site-wide signing keys +INSTANCE_KEY_RSA=/etc/fastapi-dls/instance.private.pem +INSTANCE_KEY_PUB=/etc/fastapi-dls/instance.public.pem diff --git a/.DEBIAN/fastapi-dls.service b/.DEBIAN/fastapi-dls.service new file mode 100644 index 0000000..368d494 --- /dev/null +++ b/.DEBIAN/fastapi-dls.service @@ -0,0 +1,25 @@ +[Unit] +Description=Service for fastapi-dls +Documentation=https://git.collinwebdesigns.de/oscar.krause/fastapi-dls +After=network.target + +[Service] +User=www-data +Group=www-data +AmbientCapabilities=CAP_NET_BIND_SERVICE +WorkingDirectory=/usr/share/fastapi-dls/app +EnvironmentFile=/etc/fastapi-dls/env +ExecStart=uvicorn main:app \ + --env-file /etc/fastapi-dls/env \ + --host $DLS_URL --port $DLS_PORT \ + --app-dir /usr/share/fastapi-dls/app \ + --ssl-keyfile /etc/fastapi-dls/webserver.key \ + --ssl-certfile /etc/fastapi-dls/webserver.crt \ + --proxy-headers +Restart=always +KillSignal=SIGQUIT +Type=simple +NotifyAccess=all + +[Install] +WantedBy=multi-user.target diff --git a/.DEBIAN/postinst b/.DEBIAN/postinst index d4ceee0..5624d34 100644 --- a/.DEBIAN/postinst +++ b/.DEBIAN/postinst @@ -3,89 +3,26 @@ WORKING_DIR=/usr/share/fastapi-dls CONFIG_DIR=/etc/fastapi-dls -echo "> Create config directory ..." -mkdir -p $CONFIG_DIR - -# normally we would define services in `conffiles` and as separate file, but we like to keep thinks simple. -echo "> Install service ..." -cat </etc/systemd/system/fastapi-dls.service -[Unit] -Description=Service for fastapi-dls -Documentation=https://git.collinwebdesigns.de/oscar.krause/fastapi-dls -After=network.target - -[Service] -User=www-data -Group=www-data -AmbientCapabilities=CAP_NET_BIND_SERVICE -WorkingDirectory=$WORKING_DIR/app -EnvironmentFile=$CONFIG_DIR/env -ExecStart=uvicorn main:app \\ - --env-file /etc/fastapi-dls/env \\ - --host \$DLS_URL --port \$DLS_PORT \\ - --app-dir $WORKING_DIR/app \\ - --ssl-keyfile /etc/fastapi-dls/webserver.key \\ - --ssl-certfile /etc/fastapi-dls/webserver.crt \\ - --proxy-headers -Restart=always -KillSignal=SIGQUIT -Type=simple -NotifyAccess=all - -[Install] -WantedBy=multi-user.target - -EOF - -systemctl daemon-reload - -# normally we would define configfiles in `conffiles` and as separate file, but we like to keep thinks simple. -if [[ ! -f $CONFIG_DIR/env ]]; then - echo "> Writing initial config ..." - touch $CONFIG_DIR/env - cat <$CONFIG_DIR/env -# Toggle debug mode -#DEBUG=false - -# Where the client can find the DLS server -DLS_URL=127.0.0.1 -DLS_PORT=443 - -# CORS configuration -## comma separated list without spaces -#CORS_ORIGINS="https://$DLS_URL:$DLS_PORT" - -# Lease expiration in days -LEASE_EXPIRE_DAYS=90 - -# Database location -## https://docs.sqlalchemy.org/en/14/core/engines.html -DATABASE=sqlite:///$CONFIG_DIR/db.sqlite - -# UUIDs for identifying the instance -#SITE_KEY_XID="00000000-0000-0000-0000-000000000000" -#INSTANCE_REF="00000000-0000-0000-0000-000000000000" - -# Site-wide signing keys -INSTANCE_KEY_RSA=$CONFIG_DIR/instance.private.pem -INSTANCE_KEY_PUB=$CONFIG_DIR/instance.public.pem - -EOF +if [[ ! -f $CONFIG_DIR/instance.private.pem ]]; then + echo "> Create dls-instance keypair ..." + openssl genrsa -out $CONFIG_DIR/instance.private.pem 2048 + openssl rsa -in $CONFIG_DIR/instance.private.pem -outform PEM -pubout -out $CONFIG_DIR/instance.public.pem +else + echo "> Create dls-instance keypair skipped! (exists)" fi -echo "> Create dls-instance keypair ..." -openssl genrsa -out $CONFIG_DIR/instance.private.pem 2048 -openssl rsa -in $CONFIG_DIR/instance.private.pem -outform PEM -pubout -out $CONFIG_DIR/instance.public.pem - while true; do - read -p "> Do you wish to create self-signed webserver certificate? [Y/n]" yn - yn=${yn:-y} # ${parameter:-word} If parameter is unset or null, the expansion of word is substituted. Otherwise, the value of parameter is substituted. + [[ -f $CONFIG_DIR/webserver.key ]] && default_answer="N" || default_answer="Y" + [[ $default_answer == "Y" ]] && V="Y/n" || V="y/N" + read -p "> Do you wish to create self-signed webserver certificate? [${V}]" yn + yn=${yn:-$default_answer} # ${parameter:-word} If parameter is unset or null, the expansion of word is substituted. Otherwise, the value of parameter is substituted. case $yn in [Yy]*) + echo "> Generating keypair ..." openssl req -x509 -nodes -days 3650 -newkey rsa:2048 -keyout $CONFIG_DIR/webserver.key -out $CONFIG_DIR/webserver.crt break ;; - [Nn]*) break ;; + [Nn]*) echo "> Generating keypair skipped! (exists)"; break ;; *) echo "Please answer [y] or [n]." ;; esac done diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 6f0e7a8..95b343a 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -46,7 +46,10 @@ build:apt: - cp README.md version.env build/usr/share/fastapi-dls # create conf file - mkdir -p build/etc/fastapi-dls - - touch build/etc/fastapi-dls/env + - cp .DEBIAN/env.default build/etc/fastapi-dls/env + # create service file + - mkdir -p build/etc/systemd/system + - cp .DEBIAN/fastapi-dls.service build/etc/systemd/system # cd into "build/" - cd build/ script: @@ -142,6 +145,7 @@ test: --proxy-headers & - FASTAPI_DLS_PID=$! - echo "Started service with pid $FASTAPI_DLS_PID" + - cat /etc/fastapi-dls/env # testing service - if [ "`curl --insecure -s https://127.0.0.1/-/health | jq .status`" != "up" ]; then echo "Success"; else "Error"; fi # cleanup From 52ffedffc72306edcf1ded258e9cd4e9c2654202 Mon Sep 17 00:00:00 2001 From: Oscar Krause Date: Wed, 4 Jan 2023 11:14:26 +0100 Subject: [PATCH 37/47] code styling --- .DEBIAN/postinst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.DEBIAN/postinst b/.DEBIAN/postinst index 5624d34..fbf9b82 100644 --- a/.DEBIAN/postinst +++ b/.DEBIAN/postinst @@ -52,7 +52,7 @@ cat < Date: Wed, 4 Jan 2023 17:17:58 +0100 Subject: [PATCH 38/47] README.md - moved "Endpoints" below "Setup" --- README.md | 116 +++++++++++++++++++++++++++--------------------------- 1 file changed, 58 insertions(+), 58 deletions(-) diff --git a/README.md b/README.md index e97b8df..7f5288a 100644 --- a/README.md +++ b/README.md @@ -14,64 +14,6 @@ Only the clients need a connection to this service on configured port. - check why windows guests display "can't acquire license" although in log there is no message displayed and license is also acquired successfully -## Endpoints - -### `GET /` - -Redirect to `/-/readme`. - -### `GET /-/health` - -Status endpoint, used for *healthcheck*. - -### `GET /-/config` - -Shows current runtime environment variables and their values. - -### `GET /-/readme` - -HTML rendered README.md. - -### `GET /-/docs`, `GET /-/redoc` - -OpenAPI specifications rendered from `GET /-/openapi.json`. - -### `GET /-/manage` - -Shows a very basic UI to delete origins or leases. - -### `GET /-/origins?leases=false` - -List registered origins. - -| Query Parameter | Default | Usage | -|-----------------|---------|--------------------------------------| -| `leases` | `false` | Include referenced leases per origin | - -### `DELETE /-/origins` - -Deletes all origins and their leases. - -### `GET /-/leases?origin=false` - -List current leases. - -| Query Parameter | Default | Usage | -|-----------------|---------|-------------------------------------| -| `origin` | `false` | Include referenced origin per lease | - -### `DELETE /-/lease/{lease_ref}` - -Deletes an lease. - -### `GET /-/client-token` - -Generate client token, (see [installation](#installation)). - -### Others - -There are some more internal api endpoints for handling authentication and lease process. - # Setup (Service) ## Docker @@ -338,6 +280,64 @@ Restart-Service NVDisplay.ContainerLocalSystem 'C:\Program Files\NVIDIA Corporation\NVSMI\nvidia-smi.exe' -q | Select-String "License" ``` +## Endpoints + +### `GET /` + +Redirect to `/-/readme`. + +### `GET /-/health` + +Status endpoint, used for *healthcheck*. + +### `GET /-/config` + +Shows current runtime environment variables and their values. + +### `GET /-/readme` + +HTML rendered README.md. + +### `GET /-/docs`, `GET /-/redoc` + +OpenAPI specifications rendered from `GET /-/openapi.json`. + +### `GET /-/manage` + +Shows a very basic UI to delete origins or leases. + +### `GET /-/origins?leases=false` + +List registered origins. + +| Query Parameter | Default | Usage | +|-----------------|---------|--------------------------------------| +| `leases` | `false` | Include referenced leases per origin | + +### `DELETE /-/origins` + +Deletes all origins and their leases. + +### `GET /-/leases?origin=false` + +List current leases. + +| Query Parameter | Default | Usage | +|-----------------|---------|-------------------------------------| +| `origin` | `false` | Include referenced origin per lease | + +### `DELETE /-/lease/{lease_ref}` + +Deletes an lease. + +### `GET /-/client-token` + +Generate client token, (see [installation](#installation)). + +### Others + +There are some more internal api endpoints for handling authentication and lease process. + # Troubleshoot ## Linux From 4325560ec4552ebd778f981698dd22d9670fd43a Mon Sep 17 00:00:00 2001 From: Oscar Krause Date: Wed, 4 Jan 2023 17:18:13 +0100 Subject: [PATCH 39/47] README.md - added some collapses for logs --- README.md | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/README.md b/README.md index 7f5288a..c1e127d 100644 --- a/README.md +++ b/README.md @@ -358,6 +358,9 @@ This message can be ignored. - Ref. https://github.com/encode/uvicorn/issues/441 +
+ Log example + ``` WARNING:uvicorn.error:Invalid HTTP request received. Traceback (most recent call last): @@ -376,6 +379,8 @@ Traceback (most recent call last): h11._util.RemoteProtocolError: no request line received ``` +
+ ## Windows ### Required cipher on Windows Guests (e.g. managed by domain controller with GPO) @@ -452,6 +457,9 @@ The driver wants to release current leases on shutting down windows. This endpoi is currently not implemented. The error message looks like and safely can be ignored (since we have no license limitation :P): +
+ Log example + ``` <1>:NLS initialized <1>:License acquired successfully. (Info: 192.168.178.110, NVIDIA RTX Virtual Workstation; Expiry: 2023-3-30 23:0:22 GMT) @@ -474,6 +482,8 @@ limitation :P): <0>:End Logging ``` +
+ # Credits Thanks to vGPU community and all who uses this project and report bugs. From e8736c94eca7b501bee343fcaf4e4b99f8096593 Mon Sep 17 00:00:00 2001 From: Oscar Krause Date: Wed, 4 Jan 2023 17:46:02 +0100 Subject: [PATCH 40/47] docker-compose.yml - disabled internal ssl support --- docker-compose.yml | 46 +++++++++++++++++++++++++--------------------- 1 file changed, 25 insertions(+), 21 deletions(-) diff --git a/docker-compose.yml b/docker-compose.yml index 2ebd525..b52a58a 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -2,13 +2,28 @@ version: '3.9' x-dls-variables: &dls-variables DLS_URL: localhost # REQUIRED, change to your ip or hostname - DLS_PORT: 443 # must match nginx listen port + DLS_PORT: 443 # must match nginx listen & exposed port LEASE_EXPIRE_DAYS: 90 DATABASE: sqlite:////app/database/db.sqlite DEBUG: false services: - web: + dls: + image: collinwebdesigns/fastapi-dls:latest + restart: always + environment: + <<: *dls-variables + volumes: + - /opt/docker/fastapi-dls/cert:/app/cert # instance.private.pem, instance.public.pem + - db:/app/database + entrypoint: ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8000", "--app-dir", "/app", "--proxy-headers"] + healthcheck: + test: ["CMD", "curl", "--fail", "http://localhost:8000/-/health"] + interval: 10s + timeout: 5s + retries: 3 + start_period: 30s + proxy: image: nginx ports: # thees are ports where nginx (!) is listen to @@ -17,14 +32,14 @@ services: volumes: - /opt/docker/fastapi-dls/cert:/opt/cert healthcheck: - test: [ "CMD", "curl", "--insecure", "--fail", "https://localhost/-/health" ] + test: ["CMD", "curl", "--insecure", "--fail", "https://localhost/-/health"] interval: 10s timeout: 5s retries: 3 start_period: 30s command: | - bash -c 'bash -s <<"EOF" - cat > /etc/nginx/nginx.conf <<"EON" + bash -c "bash -s <<\"EOF\" + cat > /etc/nginx/nginx.conf <<\"EON\" daemon off; user root; worker_processes auto; @@ -39,7 +54,7 @@ services: include /etc/nginx/mime.types; upstream dls-backend { - server dls:443; + server dls:8000; # must match dls listen port } server { @@ -60,18 +75,17 @@ services: ssl_prefer_server_ciphers on; location / { - proxy_ssl_verify off; proxy_set_header Host $$http_host; proxy_set_header X-Real-IP $$remote_addr; proxy_set_header X-Forwarded-For $$proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Proto $$scheme; - proxy_pass https://dls-backend$$request_uri; + proxy_pass http://dls-backend$$request_uri; } location = /-/health { access_log off; add_header 'Content-Type' 'application/json'; - return 200; # '{\"status\":\"up\",\"service\":\"nginx\"}'; + return 200 '{\"status\":\"up\",\"service\":\"nginx\"}'; } } @@ -84,12 +98,11 @@ services: server_name _; location /leasing/v1/lessor/shutdown { - proxy_ssl_verify off; proxy_set_header Host $$http_host; proxy_set_header X-Real-IP $$remote_addr; proxy_set_header X-Forwarded-For $$proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Proto $$scheme; - proxy_pass https://dls-backend/leasing/v1/lessor/shutdown; + proxy_pass http://dls-backend/leasing/v1/lessor/shutdown; } location / { @@ -99,16 +112,7 @@ services: } EON nginx - EOF' - - dls: - image: collinwebdesigns/fastapi-dls:latest - restart: always - environment: - <<: *dls-variables - volumes: - - /opt/docker/fastapi-dls/cert:/app/cert - - db:/app/database + EOF" volumes: db: From f5557a5ccd4711e341c8b5b17de72cfb7647ab94 Mon Sep 17 00:00:00 2001 From: Oscar Krause Date: Wed, 4 Jan 2023 17:46:19 +0100 Subject: [PATCH 41/47] README.md --- README.md | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-) diff --git a/README.md b/README.md index c1e127d..2343585 100644 --- a/README.md +++ b/README.md @@ -336,10 +336,12 @@ Generate client token, (see [installation](#installation)). ### Others -There are some more internal api endpoints for handling authentication and lease process. +There are many other internal api endpoints for handling authentication and lease process. # Troubleshoot +**Please make sure that fastapi-dls and your guests are on the same timezone!** + ## Linux Logs are available with `journalctl -u nvidia-gridd -f`. @@ -448,10 +450,7 @@ Dec 20 17:53:34 ubuntu-grid-server nvidia-gridd[10354]: License acquired success -### Error on releasing leases on shutdown (fixed in 1.3 by using reverse proxy) - -**UPDATE for version `1.3`**: This issue can be fixed by using a reverse proxy (e.g. `nginx`). Please read section -below. +### Error on releasing leases on shutdown (can be fixed with reverse proxy) The driver wants to release current leases on shutting down windows. This endpoint needs to be a http endpoint and is currently not implemented. The error message looks like and safely can be ignored (since we have no license From 24dba89dbecee884af8b3f298a62bfc15e36b1fc Mon Sep 17 00:00:00 2001 From: Oscar Krause Date: Wed, 4 Jan 2023 17:58:23 +0100 Subject: [PATCH 42/47] removed todos, currently all done or there is a branch for it --- README.md | 4 ---- 1 file changed, 4 deletions(-) diff --git a/README.md b/README.md index 2343585..0ff38a2 100644 --- a/README.md +++ b/README.md @@ -9,10 +9,6 @@ Only the clients need a connection to this service on configured port. [[_TOC_]] -## ToDo's - -- check why windows guests display "can't acquire license" although in log there is no message displayed and license is - also acquired successfully # Setup (Service) From 02fccb3605e08787ff54a464fe0eb9bb7e21e121 Mon Sep 17 00:00:00 2001 From: Oscar Krause Date: Wed, 4 Jan 2023 18:05:07 +0100 Subject: [PATCH 43/47] README.md --- README.md | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-) diff --git a/README.md b/README.md index 0ff38a2..6af952d 100644 --- a/README.md +++ b/README.md @@ -34,6 +34,8 @@ openssl req -x509 -nodes -days 3650 -newkey rsa:2048 -keyout $WORKING_DIR/webse **Start container** +To test if everything is set up properly you can start container as following: + ```shell docker volume create dls-db docker run -e DLS_URL=`hostname -i` -e DLS_PORT=443 -p 443:443 -v $WORKING_DIR:/app/cert -v dls-db:/app/database collinwebdesigns/fastapi-dls:latest @@ -41,7 +43,7 @@ docker run -e DLS_URL=`hostname -i` -e DLS_PORT=443 -p 443:443 -v $WORKING_DIR:/ **Docker-Compose / Deploy stack** -Goto [`docker-compose.yml`](docker-compose.yml) for more advanced example. +Goto [`docker-compose.yml`](docker-compose.yml) for more advanced example (with reverse proxy usage). ```yaml version: '3.9' @@ -446,11 +448,10 @@ Dec 20 17:53:34 ubuntu-grid-server nvidia-gridd[10354]: License acquired success -### Error on releasing leases on shutdown (can be fixed with reverse proxy) +### Error on releasing leases on shutdown (can be ignored and/or fixed with reverse proxy) -The driver wants to release current leases on shutting down windows. This endpoint needs to be a http endpoint and -is currently not implemented. The error message looks like and safely can be ignored (since we have no license -limitation :P): +The driver wants to release current leases on shutting down windows. This endpoint needs to be a http endpoint. +The error message can safely be ignored (since we have no license limitation :P) and looks like this:
Log example @@ -462,7 +463,7 @@ limitation :P): <0>:End Logging ``` -#### log with 1.3 and nginx as reverse proxy +#### log with nginx as reverse proxy (see [docker-compose.yml](docker-compose.yml)) ``` <1>:NLS initialized From aecad829143f9a0fafce3927f7dd83a4695b644b Mon Sep 17 00:00:00 2001 From: Oscar Krause Date: Wed, 4 Jan 2023 18:12:59 +0100 Subject: [PATCH 44/47] main.py - added confirmation to deleteOrigins() --- app/main.py | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/app/main.py b/app/main.py index 3bb2a3a..573cb42 100644 --- a/app/main.py +++ b/app/main.py @@ -115,14 +115,18 @@ async def _manage(request: Request): FastAPI-DLS Management - +