Commit Graph

115 Commits

Author SHA1 Message Date
VishalDalwadi e43ebb80e0 fix(go): migrate nameservers; 2026-04-01 23:44:07 +05:30
VishalDalwadi 74d309ead4 fix(go): migrate nameservers; 2026-04-01 22:53:46 +05:30
VishalDalwadi 729bec02b9 fix(go): cleanup acls in the same goroutine; 2026-04-01 17:52:03 +05:30
Abhishek Kondur 12cc967ba1 Fixes/v1.5.1 (#3938)
* fix(go): set persistent keep alive when registering host using sso;

* fix(go): run posture check violations on delete;

* fix(go): upsert node on approving pending host;

* fix(go): resolve concurrency issues during group delete cleanup;

* fix(go): update doc links;

* fix(go): add created and updated fields to host;

* fix(go): skip delete and update superadmin on sync users;

* fix(go): use conn directly for now;

* fix(go): remove acl for idp groups;

* fix(go): quote fields;

* fix(go): use filters with count;

* feat(go): add a search query;

* fix(go): cleanup acls;

* fix(go): review fixes;

* fix(go): remove additional loop;

* fix(go): fix

* v1.5.1: separate out idp sync and reset signals for HA

* v1.5.1: add grps with name for logging

* v1.5.1: clear posture check violations when all checks are deleted

* v1.5.1: set static when default host

* v1.5.1: fix db status check

* rm set max conns

* v1.5.1: reset auto assigned gw when disconnected

* fix(go): skip global network admin and user groups when splitting;

* v1.5.1: fix update node call from client

* fix(go): separate out migration from normal usage;

* fix(go): skip default groups;

* fix(go): create policies for existing groups on network create;

* fix(go): skip fatal log on clickhouse conn;

* fix(go): add posture check cleanup;

---------

Co-authored-by: VishalDalwadi <dalwadivishal26@gmail.com>
Co-authored-by: Vishal Dalwadi <51291657+VishalDalwadi@users.noreply.github.com>
2026-03-28 01:08:59 +05:30
Abhishek Kondur c3c3ed1fb8 NM-254: add bulk delete apis, rm old acl code (#3937)
* NM-254: add bulk delete apis for users, hosts, nodes and optimise postgres connection settings

* NM-254: rm debug logs

* NM-254: add bulk delete apis, remove old acl code

* NM-254: rm unused flag

* NM-254: fix bulk delete bugs, add security and performance improvements

- Fix host delete notifying peers before confirming deletion from DB
- Fix self-delete vulnerability in bulk user delete
- Fix DissasociateNodeFromHost failing when host.Nodes is empty
- Fix AssociateNodeToHost/DissasociateNodeFromHost stale read race
- Hoist GetAllExtClients outside loop in bulk user delete/status
- Move initializeUUID outside master-pod guard for HA correctness

* NM-254: return 202 Accepted for async bulk APIs, fix relay allowedIPs and host association error handling

- Change all bulk endpoints (hosts, nodes, users, ext clients) from
  200 OK to 202 Accepted to correctly signal async processing
- Add ReturnAcceptedResponse helper in logic/errors.go
- Fix GetAllowedIpsForRelayed returning empty allowedIPs slice,
  restoring relay connectivity
- Make AssociateNodeToHost and DissasociateNodeFromHost return an
  error when the host DB re-fetch fails instead of silently using
  stale data
- Add bulk-apis.md documenting all five bulk endpoints

* NM-254: rm coredns container

* NM-254: add bulk apis for node,extclient status, add activity logs to bulk apis

* NM-254: add bulk api for connection toggle

* NM-254: add network check

* Update controllers/hosts.go

Co-authored-by: tenki-reviewer[bot] <262613592+tenki-reviewer[bot]@users.noreply.github.com>

* NM-254: optimise bulk extclient deletion

---------

Co-authored-by: tenki-reviewer[bot] <262613592+tenki-reviewer[bot]@users.noreply.github.com>
2026-03-26 10:15:07 +05:30
Abhishek Kondur 79c56b0c1c NM-273: Vnat pool assignments fix (#3926)
* NM-273: Vnat pool assignments fix

* NM-273: rename var

* NM-273: add 2 vCPUs indication for monitoring stack
2026-03-20 08:10:16 +05:30
Abhishek Kondur edda2868fc NM-163: Users, Groups, Roles, Networks and Hosts Table Migration (#3910)
* feat(go): add user schema;

* feat(go): migrate to user schema;

* feat(go): add audit fields;

* feat(go): remove unused fields from the network model;

* feat(go): add network schema;

* feat(go): migrate to network schema;

* refactor(go): add comment to clarify migration logic;

* fix(go): test failures;

* fix(go): test failures;

* feat(go): change membership table to store memberships at all scopes;

* feat(go): add schema for access grants;

* feat(go): remove nameservers from new networks table; ensure db passed for schema functions;

* feat(go): set max conns for sqlite to 1;

* fix(go): issues updating user account status;

* refactor(go): remove converters and access grants;

* refactor(go): add json tags in schema models;

* refactor(go): rename file to migrate_v1_6_0.go;

* refactor(go): add user groups and user roles tables; use schema tables;

* refactor(go): inline get and list from schema package;

* refactor(go): inline get network and list users from schema package;

* fix(go): staticcheck issues;

* fix(go): remove test not in use; fix test case;

* fix(go): validate network;

* fix(go): resolve static checks;

* fix(go): new models errors;

* fix(go): test errors;

* fix(go): handle no records;

* fix(go): add validations for user object;

* fix(go): set correct extclient status;

* fix(go): test error;

* feat(go): make schema the base package;

* feat(go): add host schema;

* feat(go): use schema host everywhere;

* feat(go): inline get host, list hosts and delete host;

* feat(go): use non-ptr value;

* feat(go): use save to upsert all fields;

* feat(go): use save to upsert all fields;

* feat(go): save turn endpoint as string;

* feat(go): check for gorm error record not found;

* fix(go): test failures;

* fix(go): update all network fields;

* fix(go): update all network fields;

* feat(go): add paginated list networks api;

* feat(go): add paginated list users api;

* feat(go): add paginated list hosts api;

* feat(go): add pagination to list groups api;

* fix(go): comment;

* fix(go): implement marshal and unmarshal text for custom types;

* fix(go): implement marshal and unmarshal json for custom types;

* fix(go): just use the old model for unmarshalling;

* fix(go): implement marshal and unmarshal json for custom types;

* feat(go): remove paginated list networks api;

* feat(go): use custom paginated response object;

* fix(go): ensure default values for page and per_page are used when not passed;

* fix(go): rename v1.6.0 to v1.5.1;

* fix(go): check for gorm.ErrRecordNotFound instead of database.IsEmptyRecord;

* fix(go): use host id, not pending host id;

* feat(go): add filters to paginated apis;

* feat(go): add filters to paginated apis;

* feat(go): remove check for max username length;

* feat(go): add filters to count as well;

* feat(go): use library to check email address validity;

* feat(go): ignore pagination if params not passed;

* fix(go): pagination issues;

* fix(go): check exists before using;

* fix(go): remove debug log;

* fix(go): use gorm err record not found;

* fix(go): use gorm err record not found;

* fix(go): use user principal name when creating pending user;

* fix(go): use schema package for consts;

* fix(go): prevent disabling superadmin user;

Co-authored-by: tenki-reviewer[bot] <262613592+tenki-reviewer[bot]@users.noreply.github.com>

* fix(go): swap is admin and is superadmin;

Co-authored-by: tenki-reviewer[bot] <262613592+tenki-reviewer[bot]@users.noreply.github.com>

* fix(go): remove dead code block;

https://github.com/gravitl/netmaker/pull/3910#discussion_r2928837937

* fix(go): incorrect message when trying to disable self;

https://github.com/gravitl/netmaker/pull/3910#discussion_r2928837934

* fix(go): use correct header;

Co-authored-by: tenki-reviewer[bot] <262613592+tenki-reviewer[bot]@users.noreply.github.com>

* fix(go): return after error response;

Co-authored-by: tenki-reviewer[bot] <262613592+tenki-reviewer[bot]@users.noreply.github.com>

* fix(go): use correct order of params;

https://github.com/gravitl/netmaker/pull/3910#discussion_r2929593036

* fix(go): set default values for page and page size; use v2 instead of /list;

* Update logic/auth.go

Co-authored-by: tenki-reviewer[bot] <262613592+tenki-reviewer[bot]@users.noreply.github.com>

* Update schema/user_roles.go

Co-authored-by: tenki-reviewer[bot] <262613592+tenki-reviewer[bot]@users.noreply.github.com>

* fix(go): syntax error;

* fix(go): set default values when page and per_page are not passed or 0;

* fix(go): use uuid.parse instead of uuid.must parse;

* fix(go): review errors;

* fix(go): review errors;

* Update controllers/user.go

Co-authored-by: tenki-reviewer[bot] <262613592+tenki-reviewer[bot]@users.noreply.github.com>

* Update controllers/user.go

Co-authored-by: tenki-reviewer[bot] <262613592+tenki-reviewer[bot]@users.noreply.github.com>

* NM-163: fix errors:

* Update db/types/options.go

Co-authored-by: tenki-reviewer[bot] <262613592+tenki-reviewer[bot]@users.noreply.github.com>

* fix(go): persist return user in event;

* Update db/types/options.go

Co-authored-by: tenki-reviewer[bot] <262613592+tenki-reviewer[bot]@users.noreply.github.com>

* NM-163: duplicate lines of code

* NM-163: fix(go): fix missing return and filter parsing in user controller

- Add missing return after error response in updateUserAccountStatus
  to prevent double-response and spurious ext-client side-effects
- Use switch statements in listUsers to skip unrecognized
  account_status and mfa_status filter values

* fix(go): check for both min and max page size;

* fix(go): enclose transfer superadmin in transaction;

* fix(go): review errors;

* fix(go): remove free tier checks;

* fix(go): review fixes;

---------

Co-authored-by: VishalDalwadi <dalwadivishal26@gmail.com>
Co-authored-by: Vishal Dalwadi <51291657+VishalDalwadi@users.noreply.github.com>
Co-authored-by: tenki-reviewer[bot] <262613592+tenki-reviewer[bot]@users.noreply.github.com>
2026-03-17 19:36:52 +05:30
Vishal Dalwadi f723fc5202 NM-214: Expect GeoInfo to come from Netclient (#3833)
* feat(go): expect geoinfo from netclient;

* feat(go): add geoinfo util;
2026-02-02 14:14:49 +04:00
Abhishek Kondur c2423bf20b NM-213: JIT Network Access (#3830)
* NM-213: JIT access

* NM-213: handle expiry duration in epoch

* NM-213: avoid jit checks for admins

* NM-213: add jit expiry field to extclient

* NM-213: format jit emails

* NM-213: format jit email templates

* NM-213: update jit expiry on existing clients

* NM-213: update jit expiry on existing clients
2026-01-31 11:45:16 +04:00
Abhishek Kondur a51649a1df NM-205: Overlapping egress ranges (#3820)
* NM-205: init virtual NAT for egress ranges

* NM-205: add virtual NAT to egress peer update model

* NM-205: add virtual nat to allowed ips if present

* NM-205: fix create api, update nat fields

* NM-205: fix virtual nat range assignment

* NM-210: use virtual ranges for allowed ips for egress ips

* NM-205: allows update of nat settings

* NM-205: fix migration of egress virtual ranges

* NM-205: fix update operations

* NM-205: if domain range is set,overiride nat setting to direct

* NM-205: validate NAT pool fields on net update, add nat mode egress metric range struct

* NM-205: fix nat egress range

* NM-205: add virtual egress ranges to extclient allowed Ips

* NM-205: add overlapping egress ranges

* NM-205: match egress cidr for virtual nat if less than prefix length

* NM-205: match egress cidr for virtual nat if less than prefix length

* NM-205: force update overlapping egress feature flag

* NM-205: disable virtual nat for ipv6

* NM-205: simplify egress update ops

* NM-205: fix tests

* NM-205: NAT pool update with prefix length validation

* NM-205: send virtual egress ranges for non egress nodes

* NM-205: assign virtual NAT on creation
2026-01-29 14:36:52 +04:00
Vishal Dalwadi 33c646736c NM-212: IP Detection Interval (#3810)
* feat(go): add ip detection interval setting;

* feat(go): ensure ip detection interval is greater than 15s;
2026-01-26 15:09:15 +04:00
Abhishek Kondur 0c9ed2542b NM-202: fix egress domain routing (#3793)
* NM-195: add key tags info to posture check on join

* NM-195: add network user grps to posture check

* NM-195: add posture checks to middleware

* fix: return error when group network roles are set for specific networks and all networks;

* add all posture check to rsrc permission check func

* NM-202: fix egress domain routing

* fix: add username filter;

* feat: add fallback nameserver support;

* fix: add validation for pro as well;

* fix: skip fallback domains for user gws;

* fix: don't set domains for fallback dns servers;

* fix: validation fixes;

* fix: empty match domains for fallback nameservers;

---------

Co-authored-by: VishalDalwadi <dalwadivishal26@gmail.com>
2025-12-22 10:10:40 +04:00
Vishal Dalwadi ba9af3bfd6 Auto create default match all google dns nameserver (#3782)
* feat: auto create default match all google dns nameserver;

* fix: set column name;

* feat: prevent default ns delete;

* fix: set status;

* fix: create dns nameserver on network creation;
2025-12-16 15:58:04 +04:00
Abhishek Kondur 4853c9c1a1 NM-195: fix posture check for untagged resources (#3779)
* NM-195: fix posture check for untagged resources

* NM-195: reduce default cleanup interval

* NM-195: reduce default cleanup interval
2025-12-15 14:35:29 +04:00
Abhishek Kondur 9f430b67a1 NM-188: optimise User migration DB writes (#3774)
* NM-188: optimise user DB writes on migration

* NM-188: remove user migration debug logs

* NM-188: add debug logs for user migration funcs

* NM-188: fix migration bug

* NM-188: update migration comments in the test file

* NM-188: fix static checks

* fix: save acl src and dst;

* fix: check for all network access;

---------

Co-authored-by: VishalDalwadi <dalwadivishal26@gmail.com>
2025-12-12 10:37:05 +04:00
Abhishek Kondur eed32cd2d6 Merge pull request #3735 from gravitl/NM-166
NM-166: Device Posture Checks
2025-12-05 10:33:11 +04:00
Abhishek Kondur 94f3716fdf Merge pull request #3744 from gravitl/NM-167
NM-167: Auto delete Offline Nodes
2025-12-05 09:52:53 +04:00
Abhishek K c643a50b67 NM-159: simplify auto assign gw logic (#3719)
* force update host dns field if node is acting as inet gw

* add old acl support checks

* move auto relay migration to pro pkg

* add check to avoid unsetting relayed node

* simplify auto assign gw logic

* send auto assign update on un relay

* set checking time to latest on updates

* fix HA auto Relay logic

* add relay node metrics to peer signal

* move auto relay peer check

* publish host peer update

* check and unset unrelayed auto peers

* use relay node mutex to avoid rac condition

* reset autorelayed peers on auto assign gw
2025-11-06 14:54:09 +04:00
Vishal Dalwadi 94770a911d Merge pull request #3707 from gravitl/NM-158
NM-158: Search Domain Support
2025-11-03 10:54:28 +04:00
abhishek9686 ae6b68daee Merge branch 'NM-159' of https://github.com/gravitl/netmaker into NM-159 2025-11-02 23:50:59 +04:00
abhishek9686 26f36efdef NM-159: fix acl old version deprecation, add peer connection interval to server settings 2025-11-02 23:50:42 +04:00
Abhishek K 4dee6804f7 Merge branch 'develop' into NM-159 2025-11-02 14:08:38 +04:00
abhishek9686 d077bd0ac5 update settings to deprecate oldacls, avoid skipping deleted node update 2025-11-02 14:08:12 +04:00
Abhishek K 2ab1994886 Merge pull request #3709 from gravitl/master
Master
2025-10-31 15:04:49 +04:00
Abhishek K 0582b28129 NM-80: Logic to Deprecate Legacy Acls if unused (#3662)
* check and deprecate old acls

* add egress ranges by access to users

* add egress ranges by access to users

* merge v1.1

* resolve merge conflict
2025-10-28 09:53:42 +04:00
Abhishek K 74fef9fbc6 NM-122: Auto Relay, auto assignment of Gw (#3697)
* add auto realy handlers and logic funcs

* add pro func connectors

* Add auto relayed peer ips on peer update, set auto relay on gw creation

* add network id to signal, add autorelay nodes to peerudpate

* add autorelay peer update logic

* add nodes to peer update

* revert node model change

* reset auto relayed peers on the relay node on reset, add auto relay nodes to pull

* add logic api to update auto relay node

* add autoassigngw field to node, add logic to swith relay node in relayme udpate api

* add gw nodes to pull

* intilaise gw map

* HA relay functionality

* add autoassign gw option to enrollment key

* publish intant action to auto assign gw

* fix static checks

* unset relay if auto assign removed

* add host node model to auto relay info

* add host node model to auto relay info

* only use hostNode model for gws info

* handle autoassigned gw peer in the update

* handle autoassigned gw peer in the update

* handle peer updates for autoassigned gw peer

* unset auto assigned peer if relayed or failedovered
2025-10-28 09:53:31 +04:00
Abhishek K c5b48db2a1 NM-125: Egress HA by Latency, Allow Tags to be selected as routing peers (#3698)
* enable egress routing peers with tags

* remove tag from egress when deleted

* fix egress tag functionality

* filter duplicate egress ips

* set default stun server if unset

* add version to status api

* sync deleted node udpate host deletion
2025-10-25 23:49:21 +04:00
abhishek9686 dea257fb5d force update default dns 2025-10-10 18:44:39 +04:00
Abhishek K aa913d6870 NM-116: Acl Fixes (#3652)
* handle all resources tag on gw

* add egress domain ranges to node acls

* simplify extclient egress alloweips, handle nil acl rule

* fix static node status check for gw acls

* skip ns ip if contains network cidr

* skip ns ip if contains network cidr

* skip ns ip if contains network cidr
2025-09-18 22:27:49 +05:30
Abhishek K 9e0196126f NM-79: Domain Based Egress Routing (#3607)
* add support for egress domain routing

* add domain info to egress range

* fix egress domain update

* send peer update domain resolution update

* add egress domain update in the peer update

* use range field for domain check

* add egress domain to host pull

* add egress domain model to egress host update

* add egress domain model to egress host update

* update egress domain model on acls

* add check of range if domain is set

* sync egress domains to dns system

* add egress domain to match domain list, fix egress nat rule for domains

* fix all rsrcs comms

* fix static checks

* fix egress acls on CE

* check for all resources access on a node

* simplify egress acl rules

* merged ce and pro acl rule func

* fix uni direction acl rule for static nodes

* allow relayed nodes traffic

* resolve merge conflicts

* remove anywhere dst rule on user node acls

* fix: broadcast  user groups update for acl changes

* add egress domain ans routes to nodes

* add egress ranges to DST

* add all egress ranges for all resources

* fix DNS routing acls rules
2025-09-11 15:24:17 +05:30
Abhishek K dd2aece0de NM-103: fix all rsrcs comms (#3631)
* fix all rsrcs comms

* fix static checks

* fix egress acls on CE

* check for all resources access on a node

* simplify egress acl rules

* merged ce and pro acl rule func

* fix uni direction acl rule for static nodes

* allow relayed nodes traffic
2025-09-08 07:37:12 +05:30
Vishal Dalwadi 0eb7792c71 Merge pull request #3637 from gravitl/NM-100
v1.1.0 Release Fixes
2025-09-08 07:36:43 +05:30
abhishek9686 7688bc3ebc resolve merge conflicts 2025-08-29 11:37:27 +05:30
Vishal Dalwadi ec6e6c393a Prevent removing idp integration when oauth user is superadmin (#3589)
* feat(go): prevent removing idp integration when oauth user is superadmin.

* feat(go): add suggestion for user;

* feat(go): remove usages of boolean fields on user;

* feat(go): set boolean fields correctly, but don't use;

* fix(go): static issues;

* feat(go): add suggestion for user;
2025-08-25 10:28:53 +05:30
Abhishek K a8a0dd066c NM-44: Device Approvals for Network Join (#3579)
* add pending hosts apis, migration logic for network auto join field

* fix pending hosts logic on join

* delete pending hosts on host delete

* ignore pedning device request if host in the network already

* add peer update on host approval
2025-08-12 09:16:51 +05:30
Vishal Dalwadi e4da84aa85 NM-61: User group ACL fixes (#3546)
* feat(go): create default acl only for networks that are part of the group;

* feat(go): update acls on user group update and delete;

* feat(go): add migration for existing acls.

* feat(go): check for network roles in migration.
2025-08-08 22:17:39 +05:30
Vishal Dalwadi f8f4812851 Merge branch 'develop' into NM-37 2025-07-28 22:02:40 +05:30
Abhishek K 3a173bb4b2 Merge pull request #3568 from gravitl/master
Master
2025-07-28 18:17:17 +05:30
Vishal Dalwadi 0fea8af3e1 Merge branch 'develop' into NM-37 2025-07-28 15:23:03 +05:30
Vishal Dalwadi 31ed8c5262 Netmaker Desktop Session Duration (#3543)
* feat(go): allow different session durations for client apps;

* feat(go): assume call is from netdesk app if header absent;

* feat(go): allow header;

* feat(go): set client jwt validity duration on migration.
2025-07-27 08:29:14 +05:30
Vishal Dalwadi eb40471a6f feat(go): add support for user settings; 2025-07-21 12:14:09 +05:30
Abhishek K e517a0ac66 fix egress HA migration (#3555) 2025-07-18 12:33:21 +05:30
abhishek9686 4e2b7993eb fix merge conflicts 2025-06-24 17:00:43 +05:30
abhishek9686 f44c37e3aa fix merge conflicts 2025-06-24 17:00:10 +05:30
abhishek9686 b423b183e4 fix merge conflicts 2025-06-24 16:56:48 +05:30
Abhishek K 0a47cc5461 Merge branch 'develop' into NET-1994 2025-06-23 17:50:21 +05:30
abhishek9686 2f5acf6742 sync changes on startup, add create relay calls on defaul host 2025-06-23 06:28:43 +05:30
abhishek9686 d978de08d0 collect host localtion for graph 2025-06-12 15:47:24 +05:30
abhishek9686 cffc1d35ff reduce log retention rate to 7 days 2025-06-10 22:36:40 +05:30
abhishek9686 087b7393d6 update dns settigns on migration 2025-06-09 17:49:10 +05:30