20 Commits

Author SHA1 Message Date
Abhishek Kondur c3c3ed1fb8 NM-254: add bulk delete apis, rm old acl code (#3937)
* NM-254: add bulk delete apis for users, hosts, nodes and optimise postgres connection settings

* NM-254: rm debug logs

* NM-254: add bulk delete apis, remove old acl code

* NM-254: rm unused flag

* NM-254: fix bulk delete bugs, add security and performance improvements

- Fix host delete notifying peers before confirming deletion from DB
- Fix self-delete vulnerability in bulk user delete
- Fix DissasociateNodeFromHost failing when host.Nodes is empty
- Fix AssociateNodeToHost/DissasociateNodeFromHost stale read race
- Hoist GetAllExtClients outside loop in bulk user delete/status
- Move initializeUUID outside master-pod guard for HA correctness

* NM-254: return 202 Accepted for async bulk APIs, fix relay allowedIPs and host association error handling

- Change all bulk endpoints (hosts, nodes, users, ext clients) from
  200 OK to 202 Accepted to correctly signal async processing
- Add ReturnAcceptedResponse helper in logic/errors.go
- Fix GetAllowedIpsForRelayed returning empty allowedIPs slice,
  restoring relay connectivity
- Make AssociateNodeToHost and DissasociateNodeFromHost return an
  error when the host DB re-fetch fails instead of silently using
  stale data
- Add bulk-apis.md documenting all five bulk endpoints

* NM-254: rm coredns container

* NM-254: add bulk apis for node,extclient status, add activity logs to bulk apis

* NM-254: add bulk api for connection toggle

* NM-254: add network check

* Update controllers/hosts.go

Co-authored-by: tenki-reviewer[bot] <262613592+tenki-reviewer[bot]@users.noreply.github.com>

* NM-254: optimise bulk extclient deletion

---------

Co-authored-by: tenki-reviewer[bot] <262613592+tenki-reviewer[bot]@users.noreply.github.com>
2026-03-26 10:15:07 +05:30
Vishal Dalwadi 33c646736c NM-212: IP Detection Interval (#3810)
* feat(go): add ip detection interval setting;

* feat(go): ensure ip detection interval is greater than 15s;
2026-01-26 15:09:15 +04:00
Vishal Dalwadi a4981ffd26 NM-168: Network Flow Logs (#3754)
* feat(go): define flow events;

* feat(go): improve structure;

* feat(go): improve structure;

* feat(go): remove old flow definitions;

* feat(sql): add clickhouse init scripts;

* feat(sql): add protobuf spec;

* fix(sql): store ip as string;

* feat(go): move proto def to grpc dir;

* feat(go): use node instead of host as type; optimize protobuf defs;

* feat(go): add clickhouse db support; add endpoint to query flows;

* fix(go): fix clickhouse config;

* fix(go): use error response structure to report error;

* feat(go): pass flow logging status to netclient;

* feat(go): add peer ip identity map to host peer info;

* feat(go): remove prefix from participant obj fields;

* feat(go): add flow logs enabled field to host;

* feat(go): add filtering to get flow api;

* feat(go): fix record struct;

* feat(go): add exporter url to server config;

* feat(go): add exporter url to server config;

* feat(go): enable flow logs by default;

* feat(go): update nm-quick.sh;

* feat(go): update nm-quick.sh;

* feat(go): update nm-quick.sh;

* feat(go): update nm-quick.sh;

* feat(go): add db initialization logic;

* feat(go): filter by network id;

* fix(go): connection issue;

* fix(go): connection issue;

* fix(go): golang builder version;

* feat(go): add server settings for flow logs;

* feat(go): initialize clickhouse in pro; check for retention;

* feat(go): add exporter feature flags;

* feat(go): add grpc behind caddy;

* feat(go): expose ports correctly;

* fix(go): grpc caddyfile config;

* fix(go): publish exporter feature flags on license validation;

* fix(go): set server name for netmaker exporter;

* fix(go): set server name for netmaker exporter;

* fix(go): check for nil cancel func;

* fix(go): add flow logs field to api host;

* fix(go): add flow logs field to api host;

* fix(go): remove port from grpc setting;

* chore(go): tabs;

* feat(go): introduce egress range participant type;.

* feat(go): rename egress range to egress route for uniform language;

* feat(go): rename egress range to egress route for uniform language;

* feat: add peer addr identity map to host peer update;

* feat: add address identity map to host peer update;

* feat: add address identity map to host peer update;

* feat: set correct from and to args;

* feat: add support for filtering by node;

* feat: use corresponding base image;

* feat: update dockerfile base image version;

* fix: disable flow logs for all host when global settings are changed;
2025-12-12 14:12:00 +04:00
Abhishek Kondur eed32cd2d6 Merge pull request #3735 from gravitl/NM-166
NM-166: Device Posture Checks
2025-12-05 10:33:11 +04:00
Abhishek Kondur 94f3716fdf Merge pull request #3744 from gravitl/NM-167
NM-167: Auto delete Offline Nodes
2025-12-05 09:52:53 +04:00
abhishek9686 26f36efdef NM-159: fix acl old version deprecation, add peer connection interval to server settings 2025-11-02 23:50:42 +04:00
Abhishek K 0582b28129 NM-80: Logic to Deprecate Legacy Acls if unused (#3662)
* check and deprecate old acls

* add egress ranges by access to users

* add egress ranges by access to users

* merge v1.1

* resolve merge conflict
2025-10-28 09:53:42 +04:00
abhishek9686 773693568c resolve merge conflicts 2025-08-12 11:30:44 +05:30
Vishal Dalwadi d5cc14ce49 Patch: Okta IDP Integration (#3586)
* feat(go): add support for okta.

* feat(go): update docs link.

* feat(go): handle okta.

* feat(go): handle okta.
2025-08-07 23:05:58 +05:30
Vishal Dalwadi 99f1be4aba Merge branch 'develop' into NM-49
# Conflicts:
#	pro/auth/sync.go
2025-07-30 19:21:59 +05:30
Vishal Dalwadi 0fea8af3e1 Merge branch 'develop' into NM-37 2025-07-28 15:23:03 +05:30
Vishal Dalwadi 7965110ee1 Merge branch 'develop' into NM-49
# Conflicts:
#	models/settings.go
2025-07-28 11:26:02 +05:30
Vishal Dalwadi 31ed8c5262 Netmaker Desktop Session Duration (#3543)
* feat(go): allow different session durations for client apps;

* feat(go): assume call is from netdesk app if header absent;

* feat(go): allow header;

* feat(go): set client jwt validity duration on migration.
2025-07-27 08:29:14 +05:30
Vishal Dalwadi 8b23e9d6ae feat(go): add support for okta oauth; 2025-07-25 14:36:27 +05:30
Vishal Dalwadi eb40471a6f feat(go): add support for user settings; 2025-07-21 12:14:09 +05:30
Vishal Dalwadi 3551e8e24e NET-1996: Add Support for TOTP Authentication. (#3517)
* feat(git): ignore run configurations;

* feat(go): add support for TOTP authentication;

* fix(go): api docs;

* fix(go): static checks failing;

* fix(go): ignore mfa enforcement for user auth;

* feat(go): allow resetting mfa;

* feat(go): allow resetting mfa;

* feat(go): use library function;

* fix(go): signature;

* feat(go): allow only master user to unset user's mfa;

* feat(go): set caller when master to prevent panic;

* feat(go): make messages more user friendly;

* fix(go): run go mod tidy;

* fix(go): optimize imports;

* fix(go): return unauthorized on token expiry;

* fix(go): move mfa endpoints under username;

* fix(go): set is mfa enabled when converting;

* feat(go): allow authenticated users to use preauth apis;

* feat(go): set correct header value;

* feat(go): allow super-admins and admins to unset mfa;

* feat(go): allow user to unset mfa if not enforced;
2025-06-26 08:29:13 +05:30
Aceix 2df02f747e Merge pull request #3504 from gravitl/depracate-rac-autodisable
chore: deprecate rac autodisable flag
2025-06-24 23:43:44 +05:30
Abhishek K 810ff21165 NET-2014: add audit log retention period, add timestamp for events (#3486)
* revert inet gws from acl policies

* add egress range with metric for inet gw

* link pro inet funcs

* add timestamp params to activity apis
2025-06-06 13:19:56 +05:30
Vishal Dalwadi 614cf77b5a NET-1991: Add IDP sync functionality. (#3428)
* feat: api access tokens

* revoke all user tokens

* redefine access token api routes, add auto egress option to enrollment keys

* add server settings apis, add db table for settigs

* handle server settings updates

* switch to using settings from DB

* fix sever settings migration

* revet force migration for settings

* fix server settings database write

* fix revoked tokens to be unauthorized

* remove unused functions

* convert access token to sql schema

* switch access token to sql schema

* fix merge conflicts

* fix server settings types

* bypass basic auth setting for super admin

* add TODO comment

* feat(go): add types for idp package;

* feat(go): import azure sdk;

* feat(go): add stub for google workspace client;

* feat(go): implement azure ad client;

* feat(go): sync users and groups using idp client;

* publish peer update on settings update

* feat(go): read creds from env vars;

* feat(go): add api endpoint to trigger idp sync;

* fix(go): sync member changes;

* fix(go): handle error;

* fix(go): set correct response type;

* feat(go): support disabling user accounts;

1. Add api endpoints to enable and disable user accounts.
2. Add checks in authenticators to prevent disabled users from logging in.
3. Add checks in middleware to prevent api usage by disabled users.

* feat(go): use string slice for group members;

* feat(go): sync user account status from idp;

* feat(go): import google admin sdk;

* feat(go): add support for google workspace idp;

* feat(go): initialize idp client on sync;

* feat(go): sync from idp periodically;

* feat(go): improvements for google idp;

1. Use the impersonate package to authenticate.
2. Use Pages method to get all data.

* chore(go): import style changes from migration branch;

1. Singular file names for table schema.
2. No table name method.
3. Use .Model instead of .Table.
4. No unnecessary tagging.

* remove nat check on egress gateway request

* Revert "remove nat check on egress gateway request"

This reverts commit 0aff12a189.

* feat(go): add db middleware;

* feat(go): restore method;

* feat(go): add user access token schema;

* fix user auth api:

* re initalise oauth and email config

* feat(go): fetch idp creds from server settings;

* feat(go): add filters for users and groups;

* feat(go): skip sync from idp if disabled;

* feat(go): add endpoint to remove idp integration;

* feat(go): import all users if no filters;

* feat(go): assign service-user role on sync;

* feat(go): remove microsoft-go-sdk;

* feat(go): add display name field for user;

* fix(go): set account disabled correctly;

* fix(go): update user if display name changes;

* fix(go): remove auth provider when removing idp integration;

* fix(go): ignore display name if empty;

* feat(go): add idp sync interval setting;

* fix(go): error on invalid auth provider;

* fix(go): no error if no user on group delete;

* fix(go): check superadmin using platform role id;

* feat(go): add display name and account disabled to return user as well;

* feat(go): tidy go mod after merge;

* feat(go): reinitialize auth provider and idp sync hook;

* fix(go): merge error;

* fix(go): merge error;

* feat(go): use id as the external provider id;

* fix(go): comments;

* feat(go): add function to return pending users;

* feat(go): prevent external id erasure;

* fix(go): user and group sync errors;

* chore(go): cleanup;

* fix(go): delete only oauth users;

* feat(go): use uuid group id;

* export ipd id to in rest api

* feat(go): don't use uuid for default groups;

* feat(go): migrate group only if id not uuid;

* chore(go): go mod tidy;

---------

Co-authored-by: abhishek9686 <abhi281342@gmail.com>
Co-authored-by: Abhishek K <abhishek@netmaker.io>
Co-authored-by: the_aceix <aceixsmartx@gmail.com>
2025-05-21 13:48:15 +05:30
Abhishek K 309e4795a1 NET-1950: Persist Server Settings in the DB (#3419)
* feat: api access tokens

* revoke all user tokens

* redefine access token api routes, add auto egress option to enrollment keys

* add server settings apis, add db table for settigs

* handle server settings updates

* switch to using settings from DB

* fix sever settings migration

* revet force migration for settings

* fix server settings database write

* fix revoked tokens to be unauthorized

* remove unused functions

* convert access token to sql schema

* switch access token to sql schema

* fix merge conflicts

* fix server settings types

* bypass basic auth setting for super admin

* add TODO comment

* publish peer update on settings update

* chore(go): import style changes from migration branch;

1. Singular file names for table schema.
2. No table name method.
3. Use .Model instead of .Table.
4. No unnecessary tagging.

* remove nat check on egress gateway request

* Revert "remove nat check on egress gateway request"

This reverts commit 0aff12a189.

* feat(go): add db middleware;

* feat(go): restore method;

* feat(go): add user access token schema;

* fix user auth api:

* re initalise oauth and email config

* set verbosity

* sync auto update settings with hosts

* sync auto update settings with hosts

* mask secret and convert jwt duration to minutes

* convert jwt duration to minutes

* notify peers after settings update

* compare with curr settings before updating

* send host update to devices on auto update

---------

Co-authored-by: Vishal Dalwadi <dalwadivishal26@gmail.com>
2025-04-30 02:34:10 +04:00