1432 Commits

Author SHA1 Message Date
abhishek9686 5184f448c5 NM-311: Remove support for the -V installer flag in scripts/nm-quick.sh.
The script now always uses the default LATEST/BRANCH values and no longer
accepts user-specified release overrides.
2026-04-09 14:23:11 +05:30
abhishek9686 9e702c8fd9 Merge branch 'release-v1.5.1' of https://github.com/gravitl/netmaker into NM-311 2026-04-08 17:10:03 +05:30
abhishek9686 59a605d173 NM-311: get metrics interval from settings 2026-04-07 23:25:04 +05:30
abhishek9686 6a522ae857 v1.5.1: deduplicate egress routes 2026-04-06 05:36:51 +05:30
VishalDalwadi 95771eda51 fix(go): allow get user if admin or self; 2026-04-01 21:17:01 +05:30
VishalDalwadi 729bec02b9 fix(go): cleanup acls in the same goroutine; 2026-04-01 17:52:03 +05:30
Abhishek Kondur b97fbc69db NM-288: fix orphan nodes on bulk host deletion and refactor node cleanup (#3948)
* fix(go): set persistent keep alive when registering host using sso;

* fix(go): run posture check violations on delete;

* fix(go): upsert node on approving pending host;

* fix(go): resolve concurrency issues during group delete cleanup;

* fix(go): update doc links;

* fix(go): add created and updated fields to host;

* fix(go): skip delete and update superadmin on sync users;

* fix(go): use conn directly for now;

* fix(go): remove acl for idp groups;

* fix(go): quote fields;

* fix(go): use filters with count;

* feat(go): add a search query;

* fix(go): cleanup acls;

* fix(go): review fixes;

* fix(go): remove additional loop;

* fix(go): fix

* v1.5.1: separate out idp sync and reset signals for HA

* v1.5.1: add grps with name for logging

* v1.5.1: clear posture check violations when all checks are deleted

* v1.5.1: set static when default host

* v1.5.1: fix db status check

* rm set max conns

* v1.5.1: reset auto assigned gw when disconnected

* fix(go): skip global network admin and user groups when splitting;

* v1.5.1: fix update node call from client

* fix(go): separate out migration from normal usage;

* fix(go): skip default groups;

* fix(go): create policies for existing groups on network create;

* fix(go): skip fatal log on clickhouse conn;

* fix(go): add posture check cleanup;

* NM-288: populate relevant name for acl types for UI

* NM-288: populate grp names for posture check apis

* NM-228: add network grps api

* NM-288: add network users api

* now check each group's NetworkRoles for either the specific network ID or schema.AllNetworks (all_networks)

* NM-288: check and unassign auto gw when node is disconnected from cli

* NM-288: optimise network users api call

* NM-288: block auto assign when set to use inet gw

* NM-288: pass all network scoped get apis for platform users

* NM-288: fix orphan nodes on bulk host deletion and refactor node cleanup

- Extract cleanupNodeReferences() from DeleteNode to handle best-effort
  cleanup of relay, internet gw, failover, nameserver, ACL, and egress refs
- DisassociateAllNodesFromHost now calls cleanupNodeReferences + DeleteNodeByID
  directly, bypassing redundant host-association updates that could fail and
  leave nodes orphaned
- Track failed node deletions instead of unconditionally clearing host.Nodes;
  return error if any nodes couldn't be deleted to prevent host removal
- Fix DeleteNode returning error even when node was successfully deleted in
  the host-not-found path

* NM-288: cleanupNodeReferences(node) now only runs in the hard-delete path (when purge=true or alreadyDeleted=true), so it won't execute twice — once on soft-delete and again when the zombie manager hard-deletes.

* NM-288: add orphan deletion log

* DisassociateAllNodesFromHost now always returns nil, logging warnings for any partial failures (upsert errors or undeletable nodes). This allows RemoveHost to always proceed to h.Delete()

---------

Co-authored-by: VishalDalwadi <dalwadivishal26@gmail.com>
Co-authored-by: Vishal Dalwadi <51291657+VishalDalwadi@users.noreply.github.com>
2026-03-31 17:08:49 +05:30
Vishal Dalwadi c9fe773379 fix(go): remove node from enrollment key relay on delete and unset gw; (#3943) 2026-03-30 19:01:45 +05:30
Abhishek Kondur b1f348e71d NM-288: populate relevant name for acl types for UI (#3941)
* fix(go): set persistent keep alive when registering host using sso;

* fix(go): run posture check violations on delete;

* fix(go): upsert node on approving pending host;

* fix(go): resolve concurrency issues during group delete cleanup;

* fix(go): update doc links;

* fix(go): add created and updated fields to host;

* fix(go): skip delete and update superadmin on sync users;

* fix(go): use conn directly for now;

* fix(go): remove acl for idp groups;

* fix(go): quote fields;

* fix(go): use filters with count;

* feat(go): add a search query;

* fix(go): cleanup acls;

* fix(go): review fixes;

* fix(go): remove additional loop;

* fix(go): fix

* v1.5.1: separate out idp sync and reset signals for HA

* v1.5.1: add grps with name for logging

* v1.5.1: clear posture check violations when all checks are deleted

* v1.5.1: set static when default host

* v1.5.1: fix db status check

* rm set max conns

* v1.5.1: reset auto assigned gw when disconnected

* fix(go): skip global network admin and user groups when splitting;

* v1.5.1: fix update node call from client

* fix(go): separate out migration from normal usage;

* fix(go): skip default groups;

* fix(go): create policies for existing groups on network create;

* fix(go): skip fatal log on clickhouse conn;

* fix(go): add posture check cleanup;

* NM-288: populate relevant name for acl types for UI

* NM-288: populate grp names for posture check apis

* NM-228: add network grps api

* NM-288: add network users api

* now check each group's NetworkRoles for either the specific network ID or schema.AllNetworks (all_networks)

* NM-288: check and unassign auto gw when node is disconnected from cli

* NM-288: optimise network users api call

* NM-288: block auto assign when set to use inet gw

---------

Co-authored-by: VishalDalwadi <dalwadivishal26@gmail.com>
Co-authored-by: Vishal Dalwadi <51291657+VishalDalwadi@users.noreply.github.com>
2026-03-30 19:01:26 +05:30
Abhishek Kondur 12cc967ba1 Fixes/v1.5.1 (#3938)
* fix(go): set persistent keep alive when registering host using sso;

* fix(go): run posture check violations on delete;

* fix(go): upsert node on approving pending host;

* fix(go): resolve concurrency issues during group delete cleanup;

* fix(go): update doc links;

* fix(go): add created and updated fields to host;

* fix(go): skip delete and update superadmin on sync users;

* fix(go): use conn directly for now;

* fix(go): remove acl for idp groups;

* fix(go): quote fields;

* fix(go): use filters with count;

* feat(go): add a search query;

* fix(go): cleanup acls;

* fix(go): review fixes;

* fix(go): remove additional loop;

* fix(go): fix

* v1.5.1: separate out idp sync and reset signals for HA

* v1.5.1: add grps with name for logging

* v1.5.1: clear posture check violations when all checks are deleted

* v1.5.1: set static when default host

* v1.5.1: fix db status check

* rm set max conns

* v1.5.1: reset auto assigned gw when disconnected

* fix(go): skip global network admin and user groups when splitting;

* v1.5.1: fix update node call from client

* fix(go): separate out migration from normal usage;

* fix(go): skip default groups;

* fix(go): create policies for existing groups on network create;

* fix(go): skip fatal log on clickhouse conn;

* fix(go): add posture check cleanup;

---------

Co-authored-by: VishalDalwadi <dalwadivishal26@gmail.com>
Co-authored-by: Vishal Dalwadi <51291657+VishalDalwadi@users.noreply.github.com>
2026-03-28 01:08:59 +05:30
Abhishek Kondur c3c3ed1fb8 NM-254: add bulk delete apis, rm old acl code (#3937)
* NM-254: add bulk delete apis for users, hosts, nodes and optimise postgres connection settings

* NM-254: rm debug logs

* NM-254: add bulk delete apis, remove old acl code

* NM-254: rm unused flag

* NM-254: fix bulk delete bugs, add security and performance improvements

- Fix host delete notifying peers before confirming deletion from DB
- Fix self-delete vulnerability in bulk user delete
- Fix DissasociateNodeFromHost failing when host.Nodes is empty
- Fix AssociateNodeToHost/DissasociateNodeFromHost stale read race
- Hoist GetAllExtClients outside loop in bulk user delete/status
- Move initializeUUID outside master-pod guard for HA correctness

* NM-254: return 202 Accepted for async bulk APIs, fix relay allowedIPs and host association error handling

- Change all bulk endpoints (hosts, nodes, users, ext clients) from
  200 OK to 202 Accepted to correctly signal async processing
- Add ReturnAcceptedResponse helper in logic/errors.go
- Fix GetAllowedIpsForRelayed returning empty allowedIPs slice,
  restoring relay connectivity
- Make AssociateNodeToHost and DissasociateNodeFromHost return an
  error when the host DB re-fetch fails instead of silently using
  stale data
- Add bulk-apis.md documenting all five bulk endpoints

* NM-254: rm coredns container

* NM-254: add bulk apis for node,extclient status, add activity logs to bulk apis

* NM-254: add bulk api for connection toggle

* NM-254: add network check

* Update controllers/hosts.go

Co-authored-by: tenki-reviewer[bot] <262613592+tenki-reviewer[bot]@users.noreply.github.com>

* NM-254: optimise bulk extclient deletion

---------

Co-authored-by: tenki-reviewer[bot] <262613592+tenki-reviewer[bot]@users.noreply.github.com>
2026-03-26 10:15:07 +05:30
abhishek9686 0eb175a62b NM-283: set interface from host 2026-03-20 08:13:24 +05:30
Abhishek Kondur 79c56b0c1c NM-273: Vnat pool assignments fix (#3926)
* NM-273: Vnat pool assignments fix

* NM-273: rename var

* NM-273: add 2 vCPUs indication for monitoring stack
2026-03-20 08:10:16 +05:30
Abhishek Kondur 292af315dd NM-271: Scalability Improvements (#3921)
* feat(go): add user schema;

* feat(go): migrate to user schema;

* feat(go): add audit fields;

* feat(go): remove unused fields from the network model;

* feat(go): add network schema;

* feat(go): migrate to network schema;

* refactor(go): add comment to clarify migration logic;

* fix(go): test failures;

* fix(go): test failures;

* feat(go): change membership table to store memberships at all scopes;

* feat(go): add schema for access grants;

* feat(go): remove nameservers from new networks table; ensure db passed for schema functions;

* feat(go): set max conns for sqlite to 1;

* fix(go): issues updating user account status;

* NM-236: streamline operations in HA mode

* NM-236: only master pod should subscribe to updates from clients

* refactor(go): remove converters and access grants;

* refactor(go): add json tags in schema models;

* refactor(go): rename file to migrate_v1_6_0.go;

* refactor(go): add user groups and user roles tables; use schema tables;

* refactor(go): inline get and list from schema package;

* refactor(go): inline get network and list users from schema package;

* fix(go): staticcheck issues;

* fix(go): remove test not in use; fix test case;

* fix(go): validate network;

* fix(go): resolve static checks;

* fix(go): new models errors;

* fix(go): test errors;

* fix(go): handle no records;

* fix(go): add validations for user object;

* fix(go): set correct extclient status;

* fix(go): test error;

* feat(go): make schema the base package;

* feat(go): add host schema;

* feat(go): use schema host everywhere;

* feat(go): inline get host, list hosts and delete host;

* feat(go): use non-ptr value;

* feat(go): use save to upsert all fields;

* feat(go): use save to upsert all fields;

* feat(go): save turn endpoint as string;

* feat(go): check for gorm error record not found;

* fix(go): test failures;

* fix(go): update all network fields;

* fix(go): update all network fields;

* feat(go): add paginated list networks api;

* feat(go): add paginated list users api;

* feat(go): add paginated list hosts api;

* feat(go): add pagination to list groups api;

* fix(go): comment;

* fix(go): implement marshal and unmarshal text for custom types;

* fix(go): implement marshal and unmarshal json for custom types;

* fix(go): just use the old model for unmarshalling;

* fix(go): implement marshal and unmarshal json for custom types;

* NM-271:Import swap: compress/gzip replaced with github.com/klauspost/compress/gzip (2-4x faster, wire-compatible output). Added sync import.
Two sync.Pool variables (gzipWriterPool, bufferPool): reuse gzip.Writer and bytes.Buffer across calls instead of allocating fresh ones per publish.
compressPayload rewritten: pulls writer + buffer from pools, resets them, compresses at gzip.BestSpeed (level 1), copies the result out of the pooled buffer, and returns both objects to the pools.

* feat(go): remove paginated list networks api;

* feat(go): use custom paginated response object;

* NM-271: Improve server scalability under high host count

- Replace stdlib compress/gzip with klauspost/compress at BestSpeed and
  pool gzip writers and buffers via sync.Pool to eliminate compression
  as the dominant CPU hotspot.

- Debounce peer update broadcasts with a 500ms resettable window capped
  at 3s max-wait, coalescing rapid-fire PublishPeerUpdate calls into a
  single broadcast cycle.

- Cache HostPeerInfo (batch-refreshed by debounce worker) and
  HostPeerUpdate (stored as side-effect of each publish) so the pull API
  and peer_info API serve from pre-computed maps instead of triggering
  expensive per-host computations under thundering herd conditions.

- Warm both caches synchronously at startup before the first publish
  cycle so early pull requests are served instantly.

- Bound concurrent MQTT publishes to 5 via semaphore to prevent
  broker TCP buffer overflows that caused broken pipe disconnects.

- Remove manual Disconnect+SetupMQTT from ConnectionLostHandler and
  rely on the paho client's built-in AutoReconnect; add a 5s retry
  wait in publish() to ride out brief reconnection windows.

* NM-271: Reduce server CPU contention under high concurrent load

- Cache ServerSettings with atomic.Value to eliminate repeated DB reads
  on every pull request (was 32+ goroutines blocked on read lock)
- Batch UpdateNodeCheckin writes in memory, flush every 30s to reduce
  per-checkin write lock contention (was 88+ goroutines blocked)
- Enable SQLite WAL mode + busy_timeout and remove global dbMutex;
  let SQLite handle concurrency natively (reads no longer block writes)
- Move ResetFailedOverPeer/ResetAutoRelayedPeer to async in pull()
  handler since results don't affect the cached response
- Skip no-op UpsertNode writes in failover/relay reset functions
  (early return when node has no failover/relay state)
- Remove CheckHostPorts from hostUpdateFallback hot path
- Switch to pure-Go SQLite driver (glebarez/sqlite), set CGO_ENABLED=0

* fix(go): ensure default values for page and per_page are used when not passed;

* fix(go): rename v1.6.0 to v1.5.1;

* fix(go): check for gorm.ErrRecordNotFound instead of database.IsEmptyRecord;

* fix(go): use host id, not pending host id;

* NM-271: Revert pure-Go SQLite and FIPS disable to verify impact

Revert to CGO-based mattn/go-sqlite3 driver and re-enable FIPS to
isolate whether these changes are still needed now that the global
dbMutex has been removed and WAL mode is enabled. Keep WAL mode
pragma with mattn-compatible DSN format.

* feat(go): add filters to paginated apis;

* feat(go): add filters to paginated apis;

* feat(go): remove check for max username length;

* feat(go): add filters to count as well;

* feat(go): use library to check email address validity;

* feat(go): ignore pagination if params not passed;

* fix(go): pagination issues;

* fix(go): check exists before using;

* fix(go): remove debug log;

* NM-271: rm debug logs

* NM-271: check if caching is enabled

* NM-271: add server sync mq topic for HA mode

* NM-271: fix build

* NM-271: push metrics in batch to exproter over api

* NM-271: use basic auth for exporter metrics api

* fix(go): use gorm err record not found;

* NM-271: Add monitoring stack on demand

* NM-271: -m arg for install script should only add monitoring stack

* fix(go): use gorm err record not found;

* NM-271: update docker compose file for prometheus

* NM-271: update docker compose file for prometheus

* fix(go): use user principal name when creating pending user;

* fix(go): use schema package for consts;

* NM-236: rm duplicate network hook

* NM-271: add server topic to reset idp hooks on master node

* fix(go): prevent disabling superadmin user;

Co-authored-by: tenki-reviewer[bot] <262613592+tenki-reviewer[bot]@users.noreply.github.com>

* fix(go): swap is admin and is superadmin;

Co-authored-by: tenki-reviewer[bot] <262613592+tenki-reviewer[bot]@users.noreply.github.com>

* fix(go): remove dead code block;

https://github.com/gravitl/netmaker/pull/3910#discussion_r2928837937

* fix(go): incorrect message when trying to disable self;

https://github.com/gravitl/netmaker/pull/3910#discussion_r2928837934

* NM-271: fix stale peers on reset_failovered pull and add HTTP timeout to metrics exporter

Run the failover/relay reset synchronously in the pull handler so the
response reflects post-reset topology instead of serving stale cached
peers. Add a 30s timeout to the metrics exporter HTTP client to prevent
PushAllMetricsToExporter from blocking the Keepalive loop.

* NM-271: fix gzip pool corruption, MQTT topic mismatch, stale settings cache, and reduce redundant DB fetches

- Only return gzip.Writer to pool after successful Close to prevent
  silently malformed MQTT payloads from a previously errored writer.
- Fix serversync subscription to exact topic match since syncType is
  now in the message payload, not the topic path.
- Prevent zero-value ServerSettings from being cached indefinitely
  when the DB record is missing or unmarshal fails on startup.
- Return fetched hosts/nodes from RefreshHostPeerInfoCache so
  warmPeerCaches reuses them instead of querying the DB twice.
- Compute fresh HostPeerUpdate on reset_failovered pull instead of
  serving stale cache, and store result back for subsequent requests.

* NM-271: fix gzip writer pool leak, log checkin flush errors, and fix master pod ordinal parsing

- Reset gzip.Writer to io.Discard before returning to pool so errored
  writers are never leaked or silently reused with corrupt state.
- Track and log failed DB inserts in FlushNodeCheckins so operators
  have visibility when check-in timestamps are lost.
- Parse StatefulSet pod ordinal as integer instead of using HasSuffix
  to prevent netmaker-10 from being misidentified as master pod.

* NM-271: simplify masterpod logic

* fix(go): use correct header;

Co-authored-by: tenki-reviewer[bot] <262613592+tenki-reviewer[bot]@users.noreply.github.com>

* fix(go): return after error response;

Co-authored-by: tenki-reviewer[bot] <262613592+tenki-reviewer[bot]@users.noreply.github.com>

* fix(go): use correct order of params;

https://github.com/gravitl/netmaker/pull/3910#discussion_r2929593036

* fix(go): set default values for page and page size; use v2 instead of /list;

* NM-271: use host name

* Update mq/serversync.go

Co-authored-by: tenki-reviewer[bot] <262613592+tenki-reviewer[bot]@users.noreply.github.com>

* NM-271: fix duplicate serversynce case

* NM-271: streamline gw updates

* Update logic/auth.go

Co-authored-by: tenki-reviewer[bot] <262613592+tenki-reviewer[bot]@users.noreply.github.com>

* Update schema/user_roles.go

Co-authored-by: tenki-reviewer[bot] <262613592+tenki-reviewer[bot]@users.noreply.github.com>

* fix(go): syntax error;

* fix(go): set default values when page and per_page are not passed or 0;

* fix(go): use uuid.parse instead of uuid.must parse;

* fix(go): review errors;

* fix(go): review errors;

* Update controllers/user.go

Co-authored-by: tenki-reviewer[bot] <262613592+tenki-reviewer[bot]@users.noreply.github.com>

* Update controllers/user.go

Co-authored-by: tenki-reviewer[bot] <262613592+tenki-reviewer[bot]@users.noreply.github.com>

* NM-163: fix errors:

* Update db/types/options.go

Co-authored-by: tenki-reviewer[bot] <262613592+tenki-reviewer[bot]@users.noreply.github.com>

* fix(go): persist return user in event;

* Update db/types/options.go

Co-authored-by: tenki-reviewer[bot] <262613592+tenki-reviewer[bot]@users.noreply.github.com>

* NM-271: signal pull on ip changes

* NM-163: duplicate lines of code

* NM-163: fix(go): fix missing return and filter parsing in user controller

- Add missing return after error response in updateUserAccountStatus
  to prevent double-response and spurious ext-client side-effects
- Use switch statements in listUsers to skip unrecognized
  account_status and mfa_status filter values

* NM-271: signal pull req on node ip change

* fix(go): check for both min and max page size;

* NM-271: refresh node object before update

* fix(go): enclose transfer superadmin in transaction;

* fix(go): review errors;

* fix(go): remove free tier checks;

* fix(go): review fixes;

* NM-271: streamline ip pool ops

* NM-271: fix tests, set max idle conns

* NM-271: fix(go): fix data races in settings cache and peer update worker

- Use pointer type in atomic.Value for serverSettingsCache to avoid
  replacing the variable non-atomically in InvalidateServerSettingsCache
- Swap peerUpdateReplace flag before draining the channel to prevent
  a concurrent replacePeers=true from being consumed by the wrong cycle

---------

Co-authored-by: VishalDalwadi <dalwadivishal26@gmail.com>
Co-authored-by: Vishal Dalwadi <51291657+VishalDalwadi@users.noreply.github.com>
Co-authored-by: tenki-reviewer[bot] <262613592+tenki-reviewer[bot]@users.noreply.github.com>
2026-03-18 00:24:54 +05:30
Abhishek Kondur edda2868fc NM-163: Users, Groups, Roles, Networks and Hosts Table Migration (#3910)
* feat(go): add user schema;

* feat(go): migrate to user schema;

* feat(go): add audit fields;

* feat(go): remove unused fields from the network model;

* feat(go): add network schema;

* feat(go): migrate to network schema;

* refactor(go): add comment to clarify migration logic;

* fix(go): test failures;

* fix(go): test failures;

* feat(go): change membership table to store memberships at all scopes;

* feat(go): add schema for access grants;

* feat(go): remove nameservers from new networks table; ensure db passed for schema functions;

* feat(go): set max conns for sqlite to 1;

* fix(go): issues updating user account status;

* refactor(go): remove converters and access grants;

* refactor(go): add json tags in schema models;

* refactor(go): rename file to migrate_v1_6_0.go;

* refactor(go): add user groups and user roles tables; use schema tables;

* refactor(go): inline get and list from schema package;

* refactor(go): inline get network and list users from schema package;

* fix(go): staticcheck issues;

* fix(go): remove test not in use; fix test case;

* fix(go): validate network;

* fix(go): resolve static checks;

* fix(go): new models errors;

* fix(go): test errors;

* fix(go): handle no records;

* fix(go): add validations for user object;

* fix(go): set correct extclient status;

* fix(go): test error;

* feat(go): make schema the base package;

* feat(go): add host schema;

* feat(go): use schema host everywhere;

* feat(go): inline get host, list hosts and delete host;

* feat(go): use non-ptr value;

* feat(go): use save to upsert all fields;

* feat(go): use save to upsert all fields;

* feat(go): save turn endpoint as string;

* feat(go): check for gorm error record not found;

* fix(go): test failures;

* fix(go): update all network fields;

* fix(go): update all network fields;

* feat(go): add paginated list networks api;

* feat(go): add paginated list users api;

* feat(go): add paginated list hosts api;

* feat(go): add pagination to list groups api;

* fix(go): comment;

* fix(go): implement marshal and unmarshal text for custom types;

* fix(go): implement marshal and unmarshal json for custom types;

* fix(go): just use the old model for unmarshalling;

* fix(go): implement marshal and unmarshal json for custom types;

* feat(go): remove paginated list networks api;

* feat(go): use custom paginated response object;

* fix(go): ensure default values for page and per_page are used when not passed;

* fix(go): rename v1.6.0 to v1.5.1;

* fix(go): check for gorm.ErrRecordNotFound instead of database.IsEmptyRecord;

* fix(go): use host id, not pending host id;

* feat(go): add filters to paginated apis;

* feat(go): add filters to paginated apis;

* feat(go): remove check for max username length;

* feat(go): add filters to count as well;

* feat(go): use library to check email address validity;

* feat(go): ignore pagination if params not passed;

* fix(go): pagination issues;

* fix(go): check exists before using;

* fix(go): remove debug log;

* fix(go): use gorm err record not found;

* fix(go): use gorm err record not found;

* fix(go): use user principal name when creating pending user;

* fix(go): use schema package for consts;

* fix(go): prevent disabling superadmin user;

Co-authored-by: tenki-reviewer[bot] <262613592+tenki-reviewer[bot]@users.noreply.github.com>

* fix(go): swap is admin and is superadmin;

Co-authored-by: tenki-reviewer[bot] <262613592+tenki-reviewer[bot]@users.noreply.github.com>

* fix(go): remove dead code block;

https://github.com/gravitl/netmaker/pull/3910#discussion_r2928837937

* fix(go): incorrect message when trying to disable self;

https://github.com/gravitl/netmaker/pull/3910#discussion_r2928837934

* fix(go): use correct header;

Co-authored-by: tenki-reviewer[bot] <262613592+tenki-reviewer[bot]@users.noreply.github.com>

* fix(go): return after error response;

Co-authored-by: tenki-reviewer[bot] <262613592+tenki-reviewer[bot]@users.noreply.github.com>

* fix(go): use correct order of params;

https://github.com/gravitl/netmaker/pull/3910#discussion_r2929593036

* fix(go): set default values for page and page size; use v2 instead of /list;

* Update logic/auth.go

Co-authored-by: tenki-reviewer[bot] <262613592+tenki-reviewer[bot]@users.noreply.github.com>

* Update schema/user_roles.go

Co-authored-by: tenki-reviewer[bot] <262613592+tenki-reviewer[bot]@users.noreply.github.com>

* fix(go): syntax error;

* fix(go): set default values when page and per_page are not passed or 0;

* fix(go): use uuid.parse instead of uuid.must parse;

* fix(go): review errors;

* fix(go): review errors;

* Update controllers/user.go

Co-authored-by: tenki-reviewer[bot] <262613592+tenki-reviewer[bot]@users.noreply.github.com>

* Update controllers/user.go

Co-authored-by: tenki-reviewer[bot] <262613592+tenki-reviewer[bot]@users.noreply.github.com>

* NM-163: fix errors:

* Update db/types/options.go

Co-authored-by: tenki-reviewer[bot] <262613592+tenki-reviewer[bot]@users.noreply.github.com>

* fix(go): persist return user in event;

* Update db/types/options.go

Co-authored-by: tenki-reviewer[bot] <262613592+tenki-reviewer[bot]@users.noreply.github.com>

* NM-163: duplicate lines of code

* NM-163: fix(go): fix missing return and filter parsing in user controller

- Add missing return after error response in updateUserAccountStatus
  to prevent double-response and spurious ext-client side-effects
- Use switch statements in listUsers to skip unrecognized
  account_status and mfa_status filter values

* fix(go): check for both min and max page size;

* fix(go): enclose transfer superadmin in transaction;

* fix(go): review errors;

* fix(go): remove free tier checks;

* fix(go): review fixes;

---------

Co-authored-by: VishalDalwadi <dalwadivishal26@gmail.com>
Co-authored-by: Vishal Dalwadi <51291657+VishalDalwadi@users.noreply.github.com>
Co-authored-by: tenki-reviewer[bot] <262613592+tenki-reviewer[bot]@users.noreply.github.com>
2026-03-17 19:36:52 +05:30
Vishal Dalwadi fda970bafb Merge pull request #3891 from gravitl/NM-247
NM-247: Users should have only one config across GWs in a network
2026-03-12 20:38:15 +05:30
Abhishek Kondur 6552ae8167 Merge branch 'release-v1.5.0' into NM-258 2026-03-03 13:29:41 +04:00
abhishek9686 c344f54ce1 NM-258: add host claim verfication to token validation 2026-03-03 01:12:29 +04:00
Abhishek Kondur b5e6ca3e4c Merge pull request #3878 from gravitl/NM-256-v1.5.0-patch
v1.5.0: DB optimisations, Add Postgresql connection pool limits, add SSO cache cleanup hook
2026-02-27 13:51:16 +04:00
abhishek9686 f5c2d52066 NM-261: skip extra allowed ips for disbaled extclients 2026-02-25 18:25:06 +04:00
VishalDalwadi 0a76d6ce75 feat(go): add check for whether domain is ad domain; 2026-02-24 20:14:40 +05:30
abhishek9686 5309aa70d4 NM-258: fix host authrize func, check for token validity 2026-02-24 00:51:15 +04:00
abhishek9686 ad3af6ee7c NM-256: fix DB performance degradation with large user base and Entra IDP
- Replace full table scan in FetchRecord with indexed single-key lookup
      (SELECT WHERE key = ?) for PostgreSQL, SQLite, and RQLite backends
    - Add in-memory user cache (gated behind CACHING_ENABLED) to eliminate
      DB round-trips on the auth hot path (GetUser called per API request)
    - Configure PostgreSQL connection pool limits (max open/idle conns,
      conn lifetime) to prevent connection churn under load
    - Add periodic cleanup of expired SSO state entries to prevent
      unbounded table growth
    - Route GitHub OAuth user rename through standard logic functions
      to keep user cache consistent
2026-02-23 13:56:50 +04:00
abhishek9686 c29b23ccf8 v1.5.0: fix egress update 2026-02-12 22:16:02 +04:00
abhishek9686 787a00b96d v1.5.0: fix user activity permissions, fix removing nodes from egress models 2026-02-12 17:21:05 +04:00
Abhishek Kondur e475c1d57d NM-209: allow darwin clients to use inetgw, update email templates to use logo in png format (#3860)
* NM-209: allow darwin clients to use inetgw

* update email templates to use logo in png format

* NM-209: fix email logo

* NM-209: add spacing
2026-02-11 01:39:32 +04:00
Abhishek Kondur b935e3795a NM-228: add middleware checks for activity logs, remove all users get… (#3853)
* NM-228: add middleware checks for activity logs, remove all users get access platform users

* NM-228: fix middleware checks for platform users

* NM-228: add middleware check for user activity
2026-02-10 21:13:55 +04:00
Vishal Dalwadi 73949b39dd Set location if not set on host (#3859)
* fix(go): set location if not set on host;

* fix(go): set location if not set on host;

* fix(go): set location if not set on host;

* fix(go): set location if not set on host;
2026-02-10 19:25:28 +04:00
Vishal Dalwadi ba6a062a2c fix(go): set metrics port; (#3858) 2026-02-10 12:16:00 +04:00
Vishal Dalwadi 768fcc938d Fixes: V1.5.0 Don't store default domain in dns record (#3849)
* fix(go): send match domains and search domains for igws;

* fix(go): use correct datatypes for coordinates;

* fix(go): don't store default domain in dns record;

* feat(go): remove back to login button;

* feat(go): update images;
2026-02-09 14:50:08 +04:00
Vishal Dalwadi 743c3924b0 feat(go): let users control match all nameservers; (#3851) 2026-02-09 14:49:44 +04:00
Abhishek Kondur 94e0ea741d update host geo location info (#3850) 2026-02-05 16:18:19 +04:00
abhishek9686 82d3cf133c v1.4.0: resolve merge conflicts 2026-02-04 13:10:32 +04:00
Vishal Dalwadi f723fc5202 NM-214: Expect GeoInfo to come from Netclient (#3833)
* feat(go): expect geoinfo from netclient;

* feat(go): add geoinfo util;
2026-02-02 14:14:49 +04:00
Abhishek Kondur c2423bf20b NM-213: JIT Network Access (#3830)
* NM-213: JIT access

* NM-213: handle expiry duration in epoch

* NM-213: avoid jit checks for admins

* NM-213: add jit expiry field to extclient

* NM-213: format jit emails

* NM-213: format jit email templates

* NM-213: update jit expiry on existing clients

* NM-213: update jit expiry on existing clients
2026-01-31 11:45:16 +04:00
Abhishek Kondur a51649a1df NM-205: Overlapping egress ranges (#3820)
* NM-205: init virtual NAT for egress ranges

* NM-205: add virtual NAT to egress peer update model

* NM-205: add virtual nat to allowed ips if present

* NM-205: fix create api, update nat fields

* NM-205: fix virtual nat range assignment

* NM-210: use virtual ranges for allowed ips for egress ips

* NM-205: allows update of nat settings

* NM-205: fix migration of egress virtual ranges

* NM-205: fix update operations

* NM-205: if domain range is set,overiride nat setting to direct

* NM-205: validate NAT pool fields on net update, add nat mode egress metric range struct

* NM-205: fix nat egress range

* NM-205: add virtual egress ranges to extclient allowed Ips

* NM-205: add overlapping egress ranges

* NM-205: match egress cidr for virtual nat if less than prefix length

* NM-205: match egress cidr for virtual nat if less than prefix length

* NM-205: force update overlapping egress feature flag

* NM-205: disable virtual nat for ipv6

* NM-205: simplify egress update ops

* NM-205: fix tests

* NM-205: NAT pool update with prefix length validation

* NM-205: send virtual egress ranges for non egress nodes

* NM-205: assign virtual NAT on creation
2026-01-29 14:36:52 +04:00
Vishal Dalwadi d476e79278 NM-220: Flow Logs Improvements (#3812)
* feat(go): add name to src and dst info;

* feat(go): populate names to share with netclient;

* feat(go): add hostname to flow event;
2026-01-28 09:05:51 +04:00
Vishal Dalwadi d8515f2ea9 feat(go): expect deployment mode on license validation; return deployment mode when reporting tenant status; (#3809) 2026-01-26 15:10:31 +04:00
Vishal Dalwadi 33c646736c NM-212: IP Detection Interval (#3810)
* feat(go): add ip detection interval setting;

* feat(go): ensure ip detection interval is greater than 15s;
2026-01-26 15:09:15 +04:00
VishalDalwadi e4328ab546 feat(go): add global nameservers only if no match all nameserver configured; 2026-01-22 21:23:07 +05:30
Abhishek Kondur 0c9ed2542b NM-202: fix egress domain routing (#3793)
* NM-195: add key tags info to posture check on join

* NM-195: add network user grps to posture check

* NM-195: add posture checks to middleware

* fix: return error when group network roles are set for specific networks and all networks;

* add all posture check to rsrc permission check func

* NM-202: fix egress domain routing

* fix: add username filter;

* feat: add fallback nameserver support;

* fix: add validation for pro as well;

* fix: skip fallback domains for user gws;

* fix: don't set domains for fallback dns servers;

* fix: validation fixes;

* fix: empty match domains for fallback nameservers;

---------

Co-authored-by: VishalDalwadi <dalwadivishal26@gmail.com>
2025-12-22 10:10:40 +04:00
Abhishek Kondur ea75901f8f NM-195: cleanup deleted tags and user groups from posturechecks, add nameserver to middleware (#3794)
* NM-195: add key tags info to posture check on join

* NM-195: add network user grps to posture check

* NM-195: add posture checks to middleware

* add all posture check to rsrc permission check func

* NM-195: cleanup deleted tags and user groups from posturechecks

* NM-195: add Nameserver to middleware
2025-12-19 17:36:39 +04:00
Abhishek Kondur 79dd46c8df NM-195: Add posture check to middleware (#3791)
* NM-195: add key tags info to posture check on join

* NM-195: add network user grps to posture check

* NM-195: add posture checks to middleware

* add all posture check to rsrc permission check func
2025-12-19 09:51:32 +04:00
Vishal Dalwadi ba9af3bfd6 Auto create default match all google dns nameserver (#3782)
* feat: auto create default match all google dns nameserver;

* fix: set column name;

* feat: prevent default ns delete;

* fix: set status;

* fix: create dns nameserver on network creation;
2025-12-16 15:58:04 +04:00
Vishal Dalwadi a4981ffd26 NM-168: Network Flow Logs (#3754)
* feat(go): define flow events;

* feat(go): improve structure;

* feat(go): improve structure;

* feat(go): remove old flow definitions;

* feat(sql): add clickhouse init scripts;

* feat(sql): add protobuf spec;

* fix(sql): store ip as string;

* feat(go): move proto def to grpc dir;

* feat(go): use node instead of host as type; optimize protobuf defs;

* feat(go): add clickhouse db support; add endpoint to query flows;

* fix(go): fix clickhouse config;

* fix(go): use error response structure to report error;

* feat(go): pass flow logging status to netclient;

* feat(go): add peer ip identity map to host peer info;

* feat(go): remove prefix from participant obj fields;

* feat(go): add flow logs enabled field to host;

* feat(go): add filtering to get flow api;

* feat(go): fix record struct;

* feat(go): add exporter url to server config;

* feat(go): add exporter url to server config;

* feat(go): enable flow logs by default;

* feat(go): update nm-quick.sh;

* feat(go): update nm-quick.sh;

* feat(go): update nm-quick.sh;

* feat(go): update nm-quick.sh;

* feat(go): add db initialization logic;

* feat(go): filter by network id;

* fix(go): connection issue;

* fix(go): connection issue;

* fix(go): golang builder version;

* feat(go): add server settings for flow logs;

* feat(go): initialize clickhouse in pro; check for retention;

* feat(go): add exporter feature flags;

* feat(go): add grpc behind caddy;

* feat(go): expose ports correctly;

* fix(go): grpc caddyfile config;

* fix(go): publish exporter feature flags on license validation;

* fix(go): set server name for netmaker exporter;

* fix(go): set server name for netmaker exporter;

* fix(go): check for nil cancel func;

* fix(go): add flow logs field to api host;

* fix(go): add flow logs field to api host;

* fix(go): remove port from grpc setting;

* chore(go): tabs;

* feat(go): introduce egress range participant type;.

* feat(go): rename egress range to egress route for uniform language;

* feat(go): rename egress range to egress route for uniform language;

* feat: add peer addr identity map to host peer update;

* feat: add address identity map to host peer update;

* feat: add address identity map to host peer update;

* feat: set correct from and to args;

* feat: add support for filtering by node;

* feat: use corresponding base image;

* feat: update dockerfile base image version;

* fix: disable flow logs for all host when global settings are changed;
2025-12-12 14:12:00 +04:00
Abhishek Kondur 9f430b67a1 NM-188: optimise User migration DB writes (#3774)
* NM-188: optimise user DB writes on migration

* NM-188: remove user migration debug logs

* NM-188: add debug logs for user migration funcs

* NM-188: fix migration bug

* NM-188: update migration comments in the test file

* NM-188: fix static checks

* fix: save acl src and dst;

* fix: check for all network access;

---------

Co-authored-by: VishalDalwadi <dalwadivishal26@gmail.com>
2025-12-12 10:37:05 +04:00
Abhishek Kondur 4177348f12 NM-166: add user groups to posture checks (#3766)
* NM-166: add device posture checks apis and logic

* NM-166: add severity level to posture check and api to fetch all available attributes

* NM-166: register posture check schema

* add periodic posture check evaluation of nodes

* NM-166: add os family and kernel version to host model for linux

* add posture check violations on host registration, api to fetch node violations

* NM-166: trigger posture checks on posture checks updates

* NM-166: add version clean func

* NM-166: add allowed values for posture check attributes to api

* NM-166: format violation messages

* NM-166: fix static check

* NM-166: fix static check

* NM-166: add OS info to update extclient api

* NM-166: add sysinfo funcs

* set if only new values are not empty

* format client location

* fix posture violation for static nodes

* skip non user nodes from posture checks

* NM-166: check posture checks by tags

* NM-166: set host location before posture check

* validate posture checks by OR condition

* run posture check violation on node update

* NM-166: allow join on unviolated networks

* NM-166: update response message when posture checks are violated

* NM-166: fix static check

* NM-166: add mutex for posture check runs

* NM-166: add OS family fields to api host model

* NM-166: run posture eval for gateway ops

* NM-166: add user groups to posture checks

* NM-166: add default all user grp

* NM-166: fix posture check eval for users

* NM-166: handle user nodes in the posture checks

* NM-166: fix posture check for new config

* NM-166: skip auto update check on users and show violation on disbaled static nodes

* NM-166: add min verison check for attr

* NM-166: fix static check

* NM-166: add default admin groups
2025-12-11 12:19:30 +04:00
Vishal Dalwadi b48c705c67 Allow updating auto-assign-gw on enrollment keys (#3768)
* feat(go): allow updating auto-assign-gw on enrollment keys;

* feat: prevent auto assign gateway if relay is set;
2025-12-11 10:39:44 +04:00
Abhishek Kondur 3bab218a2b NM-175: add ipv4/ipv6 to peer info (#3763)
* add ipv4 and ipv6 addr to peer info

* NM-175: add extclient addr4 and addr6

* add user info to peerinfo
2025-12-05 18:10:09 +04:00
Abhishek Kondur eed32cd2d6 Merge pull request #3735 from gravitl/NM-166
NM-166: Device Posture Checks
2025-12-05 10:33:11 +04:00