Compare commits

...
27 Commits
Author SHA1 Message Date
fluzzi32 693531ed57 docs: regenerate pdoc HTML documentation for v6.5.0b1 2026-11-17 19:16:11 -03:00
fluzzi32 b3925fee07 refactor(spec-05-auth-fallback-chain): senior (eng_flash_high) review refinements 2026-11-17 18:30:32 -03:00
fluzzi32 1b50c7d3e7 feat(spec-05-auth-fallback-chain): worker (eng_flash_medium) implementation 2026-11-17 18:27:57 -03:00
fluzzi32 fc27747c19 feat(spec-05-auth-fallback-chain): worker (eng_flash_medium) implementation 2026-11-17 18:19:51 -03:00
fluzzi32 175e623d99 feat(spec-05-auth-fallback-chain): worker (eng_flash_medium) implementation 2026-11-17 18:01:59 -03:00
fluzzi32 ecaa836b98 refactor(spec-05-auth-fallback-chain): senior (eng_flash_high) review refinements 2026-11-17 16:33:25 -03:00
fluzzi32 f86ce21ee0 feat(spec-05-auth-fallback-chain): worker (eng_flash_medium) implementation 2026-11-17 16:31:05 -03:00
fluzzi32 dddeaf4b03 feat(spec-05-auth-fallback-chain): worker (eng_flash_high) implementation 2026-11-17 14:44:05 -03:00
fluzzi32 441bf54ac1 chore(release): bump version to v6.5.0b1 and update documentation 2026-11-17 13:44:52 -03:00
fluzzi32 c6f260efc9 feat(spec-05-auth-fallback-chain): worker (eng_flash_high) implementation 2026-11-17 12:58:00 -03:00
fluzzi32 d2da49221f Merge feature/spec-05-auth-fallback-chain: add auth fallback config flags to completion tree 2026-11-17 12:13:37 -03:00
fluzzi32 59640d20a4 feat(spec-05-auth-fallback-chain): worker (eng_flash_high) implementation 2026-11-17 12:11:02 -03:00
fluzzi32 ad9dd27316 Merge feature/spec-05-auth-fallback-chain: unattended AAA fallback chain & compose credentials 2026-11-17 11:39:38 -03:00
fluzzi32 89ca1ec621 feat(spec-05-auth-fallback-chain): worker (eng_flash_high) implementation 2026-11-17 11:35:54 -03:00
fluzzi32 d96d8fb1fa feat(spec-05-auth-fallback-chain): worker (eng_flash_high) implementation 2026-11-16 23:03:32 -03:00
fluzzi32 79d28ee6c3 feat(spec-05-auth-fallback-chain): worker (eng_flash_high) implementation 2026-11-16 22:46:44 -03:00
fluzzi32 ef6d1da5e5 Merge feature/spec-04-adaptive-timeouts-ssm: adaptive SSM/jumphost timeouts & hardened expects 2026-11-16 21:25:41 -03:00
fluzzi32 8e388cc6b6 feat(spec-04-adaptive-timeouts-ssm): worker (eng_flash_high) implementation 2026-11-16 21:14:58 -03:00
fluzzi32 c3995ea6a9 feat(spec-04-adaptive-timeouts-ssm): worker (eng_flash_high) implementation 2026-11-16 21:06:07 -03:00
fluzzi32 951961a5d3 chore(release): bump version to v6.4.0b4 and update documentation 2026-11-09 17:30:43 -03:00
fluzzi32 1caa286e65 chore(release): bump version to v6.4.0b3 and update documentation 2026-11-09 16:14:19 -03:00
fluzzi32 388d042eba Merge feature/spec-03-mcp-server: Native MCP Server (stdio/sse) with 3-tier security 2026-11-09 16:08:02 -03:00
fluzzi32 91c6732719 Merge feature/spec-02-ai-and-hooks-bugfixes: AI copilot fixes & per-instance/tenant isolation 2026-11-09 16:07:44 -03:00
fluzzi32 a2ca91dfc9 Merge feature/spec-01-security-yaml-and-cmd: YAML safe_load & command option sanitization 2026-11-09 16:07:22 -03:00
fluzzi32 2721e39e78 feat(mcp): implement native ConnpyMCPServer with stdio/SSE transports and tiered security (spec-03) 2026-11-09 15:59:50 -03:00
fluzzi32 494a7dffe6 fix(ai,hooks,mcp): resolve AI bugfixes, MethodHook instance isolation, and MCP multi-tenancy (spec-02) 2026-11-09 15:59:46 -03:00
fluzzi32 a90e8bf597 feat(security): harden yaml safe_load and connection command injection 2026-11-09 15:45:30 -03:00
65 changed files with 23480 additions and 4003 deletions

No files matched your search

+21 -4
View File
@@ -3,7 +3,7 @@
</p>
# Connpy (v6.4.0b2)
# Connpy (v6.5.0b1)
[![](https://img.shields.io/pypi/v/connpy.svg?style=flat-square)](https://pypi.org/pypi/connpy/)
[![](https://img.shields.io/pypi/pyversions/connpy.svg?style=flat-square)](https://pypi.org/pypi/connpy/)
[![](https://img.shields.io/pypi/dm/connpy.svg?style=flat-square&cacheSeconds=86400)](https://pypi.org/pypi/connpy/)
@@ -40,10 +40,18 @@ Start a standalone persistent session with the AI Copilot. Manage sessions using
conn ai "how do i check bgp summary on cisco?"
```
### 1c. MCP Integration
Connect to external data sources and tools dynamically via the Model Context Protocol (MCP). Use the interactive wizard or command actions to configure MCP servers:
### 1c. MCP Integration (Client & Server)
Connect to external data sources and tools dynamically, or expose `connpy` as a native MCP server for external agents (Claude Desktop, Cursor, IDEs):
```bash
conn ai --mcp
# MCP Client: Manage connected MCP servers
conn ai --mcp # Interactive configuration wizard
conn ai --mcp list # List configured MCP servers
conn ai --mcp add <name> <url> [os] # Add a remote SSE MCP server
# MCP Server: Serve connpy inventory, diagnostic tools, and playbooks
conn ai --mcp serve # Standard I/O mode (default, for local clients)
conn ai --mcp serve --sse --port 8080 # Remote SSE mode
conn ai --mcp serve --trust # Enable inventory mutations (CRUD) & configuration commands
```
### 1d. Local Interactive Shell (conn shell)
@@ -129,6 +137,12 @@ Customize connection settings dynamically using tags. Configure per-node setting
tags: { "os": "cisco_ios", "prompt": ".*#", "screen_length_command": "terminal length 0" }
```
### 3f. Credential Fallback Chain, `compose:` & Auth Budgets
Configure automatic backup credentials (`fallback`) on nodes or `@profile` templates so both interactive (`conn <node>`) and unattended (`conn run` / MCP) sessions switch seamlessly when primary TACACS+/AAA authentication fails (opt-out via `tags.allow_fallback: false`).
* **Fallback Formats**: Accepts `@profile`, explicit JSON list/dict entries (`[{"user": "backup", "password": "..."}]`), or `compose:` expressions. Nodes referencing multiple profiles inherit `fallback` with priority from `user`/`password` `@profile` references.
* **Password Composition (`compose:`)**: Concatenate secrets and literals explicitly using `compose:@prof1+@prof2+"suffix"` (use `@@` to escape leading literal `@`).
* **Anti-Lockout Guardrails (`conn config`)**: Enforce strict attempt bounds via `--max-auth-prompts`, `--max-total-auth-attempts`, `--max-hop-host-attempts`, `--max-global-user-attempts`, and `--disable-fallback-on-insecure-hostkey`.
---
@@ -149,6 +163,9 @@ Support for single or chained intermediate gateway nodes (SSH, Mosh, SSM, kubect
### 4c. Debug Mode, Keepalive, Logging
Track connection steps (`conn --debug node`), set idle keepalive intervals (`conn config --keepalive <seconds>`), or define dynamic output log files using variables like `${unique}`, `${host}`, `${port}`, `${user}`, `${protocol}`, or `${date 'format'}`.
### 4d. Adaptive Multi-Hop Timeouts
Differentiate `--connect-timeout` and `--auth-timeout` stages (`conn run` / `conn config`), automatically computing topological per-hop latency budgets for chained `SSM` and `Jumphost` tunnels with bounded progress extensions on slow handshakes.
---
+21 -4
View File
@@ -5,7 +5,7 @@
</p>
# Connpy (v6.4.0b2)
# Connpy (v6.5.0b1)
[![](https://img.shields.io/pypi/v/connpy.svg?style=flat-square)](https://pypi.org/pypi/connpy/)
[![](https://img.shields.io/pypi/pyversions/connpy.svg?style=flat-square)](https://pypi.org/pypi/connpy/)
[![](https://img.shields.io/pypi/dm/connpy.svg?style=flat-square&cacheSeconds=86400)](https://pypi.org/pypi/connpy/)
@@ -42,10 +42,18 @@ Start a standalone persistent session with the AI Copilot. Manage sessions using
conn ai "how do i check bgp summary on cisco?"
```
### 1c. MCP Integration
Connect to external data sources and tools dynamically via the Model Context Protocol (MCP). Use the interactive wizard or command actions to configure MCP servers:
### 1c. MCP Integration (Client & Server)
Connect to external data sources and tools dynamically, or expose `connpy` as a native MCP server for external agents (Claude Desktop, Cursor, IDEs):
```bash
conn ai --mcp
# MCP Client: Manage connected MCP servers
conn ai --mcp # Interactive configuration wizard
conn ai --mcp list # List configured MCP servers
conn ai --mcp add <name> <url> [os] # Add a remote SSE MCP server
# MCP Server: Serve connpy inventory, diagnostic tools, and playbooks
conn ai --mcp serve # Standard I/O mode (default, for local clients)
conn ai --mcp serve --sse --port 8080 # Remote SSE mode
conn ai --mcp serve --trust # Enable inventory mutations (CRUD) & configuration commands
```
### 1d. Local Interactive Shell (conn shell)
@@ -131,6 +139,12 @@ Customize connection settings dynamically using tags. Configure per-node setting
tags: { "os": "cisco_ios", "prompt": ".*#", "screen_length_command": "terminal length 0" }
```
### 3f. Credential Fallback Chain, `compose:` & Auth Budgets
Configure automatic backup credentials (`fallback`) on nodes or `@profile` templates so both interactive (`conn <node>`) and unattended (`conn run` / MCP) sessions switch seamlessly when primary TACACS+/AAA authentication fails (opt-out via `tags.allow_fallback: false`).
* **Fallback Formats**: Accepts `@profile`, explicit JSON list/dict entries (`[{"user": "backup", "password": "..."}]`), or `compose:` expressions. Nodes referencing multiple profiles inherit `fallback` with priority from `user`/`password` `@profile` references.
* **Password Composition (`compose:`)**: Concatenate secrets and literals explicitly using `compose:@prof1+@prof2+"suffix"` (use `@@` to escape leading literal `@`).
* **Anti-Lockout Guardrails (`conn config`)**: Enforce strict attempt bounds via `--max-auth-prompts`, `--max-total-auth-attempts`, `--max-hop-host-attempts`, `--max-global-user-attempts`, and `--disable-fallback-on-insecure-hostkey`.
---
@@ -151,6 +165,9 @@ Support for single or chained intermediate gateway nodes (SSH, Mosh, SSM, kubect
### 4c. Debug Mode, Keepalive, Logging
Track connection steps (`conn --debug node`), set idle keepalive intervals (`conn config --keepalive <seconds>`), or define dynamic output log files using variables like `${unique}`, `${host}`, `${port}`, `${user}`, `${protocol}`, or `${date 'format'}`.
### 4d. Adaptive Multi-Hop Timeouts
Differentiate `--connect-timeout` and `--auth-timeout` stages (`conn run` / `conn config`), automatically computing topological per-hop latency budgets for chained `SSM` and `Jumphost` tunnels with bounded progress extensions on slow handshakes.
---
+1 -1
View File
@@ -1 +1 @@
__version__ = "6.4.0b2"
__version__ = "6.5.0b1"
+16 -5
View File
@@ -1,3 +1,5 @@
#!/usr/bin/env python3
# ruff: noqa
import os
import secrets
@@ -197,6 +199,9 @@ class ai:
self.session_path = os.path.join(self.sessions_dir, f"{self.session_id}.json") if self.session_id else None
# Agnostic base prompts
self._update_base_prompts()
def _update_base_prompts(self):
architect_instructions = ""
if self.has_architect:
architect_instructions = """
@@ -237,7 +242,7 @@ class ai:
- NO WANDERING: Do not speculate. If stuck, report attempts.
- SAFETY: When you use 'run_commands' with configuration commands, the system automatically prompts the user for confirmation. Just execute - don't ask permission first.
{architect_instructions}
Network Context: {{self.long_term_memory if self.long_term_memory else "Empty."}}
Network Context: {self.long_term_memory if self.long_term_memory else "Empty."}
""").strip()
self._architect_base_prompt = dedent(f"""
@@ -277,6 +282,7 @@ class ai:
@property
def engineer_system_prompt(self):
"""Build engineer system prompt with plugin extensions."""
self._update_base_prompts()
if self.engineer_prompt_extensions:
extensions = "\n".join(self.engineer_prompt_extensions)
return self._engineer_base_prompt + f"\n\nPlugin Capabilities:\n{extensions}"
@@ -285,6 +291,7 @@ class ai:
@property
def architect_system_prompt(self):
"""Build architect system prompt with plugin extensions."""
self._update_base_prompts()
prompt = self._architect_base_prompt
if getattr(self, "one_shot", False):
prompt += "\n\nCRITICAL 1-SHOT DIAGNOSTICS DIRECTIVE:\nYou are running in a 1-shot offline diagnostics mode. There is no active conversation loop, and you are NOT conversing with a Network Engineer. You MUST deliver your complete strategic analysis immediately and directly to the user. Do not suggest or attempt to delegate/return control to the engineer."
@@ -624,6 +631,7 @@ class ai:
# Reload memory after update
with open(self.memory_path, "r") as f:
self.long_term_memory = f.read()
self._update_base_prompts()
return "Memory updated successfully."
except PermissionError as e:
@@ -859,7 +867,7 @@ class ai:
# Silently fail for LLM tools
pass
if self.architect_key:
if self.has_architect:
base_tools.extend([
{"type": "function", "function": {"name": "consult_architect", "description": "Ask the Strategic Reasoning Engine for advice on complex design, architecture, or troubleshooting decisions. You remain in control and will present the response to the user. Use this for: configuration planning, design validation, complex troubleshooting.", "parameters": {"type": "object", "properties": {"question": {"type": "string", "description": "Strategic question or decision needed."}, "technical_summary": {"type": "string", "description": "Technical findings and context gathered so far."}}, "required": ["question", "technical_summary"]}}},
{"type": "function", "function": {"name": "escalate_to_architect", "description": "Transfer full control to the Strategic Reasoning Engine. Use ONLY when the user explicitly requests the Architect or when the problem requires strategic oversight beyond consultation. After escalation, the Architect takes over the conversation.", "parameters": {"type": "object", "properties": {"reason": {"type": "string", "description": "Why you're escalating (e.g. 'User requested Architect', 'Complex multi-site design needed')."}, "context": {"type": "string", "description": "Full context and findings to hand over."}}, "required": ["reason", "context"]}}}
@@ -1216,14 +1224,17 @@ class ai:
if status: update_status("[architect]Engineer consulting Architect...")
try:
# Consultation only - Engineer stays in control
auth_kwargs = dict(self.architect_auth or {})
if self.architect_key and "api_key" not in auth_kwargs:
auth_kwargs["api_key"] = self.architect_key
claude_resp = completion(
model=self.architect_model,
messages=[
{"role": "system", "content": self.architect_system_prompt},
{"role": "user", "content": f"The Engineer needs your strategic advice.\n\nTECHNICAL SUMMARY: {args['technical_summary']}\n\nQUESTION: {args['question']}\n\nProvide strategic guidance. The Engineer will continue handling the user."}
],
api_key=self.architect_key,
num_retries=3
num_retries=3,
**auth_kwargs
)
obs = claude_resp.choices[0].message.content
if debug:
@@ -1790,7 +1801,7 @@ class PlaybookBuilderAgent:
import yaml
try:
# 1. Parse YAML
data = yaml.load(playbook_yaml, Loader=yaml.FullLoader)
data = yaml.safe_load(playbook_yaml)
except Exception as e:
return {"valid": False, "error": f"YAML Syntax Error: {e}"}
+1688
View File
File diff suppressed because it is too large. Load diff
+54 -11
View File
@@ -1,7 +1,8 @@
import sys
from rich.panel import Panel
from rich.rule import Rule
from rich.prompt import Prompt
from rich.rule import Rule
from .. import printer
@@ -36,7 +37,7 @@ class AIHandler:
try:
self.app.services.ai.delete_session(args.delete_session[0])
printer.success(f"Session {args.delete_session[0]} deleted.")
except Exception as e:
except Exception as e: # noqa: BLE001
printer.error(str(e))
return
@@ -189,7 +190,7 @@ class AIHandler:
try:
self.app.services.ai.configure_mcp(name, url=url, auto_load_on_os=os_filter)
printer.success(f"MCP server '{name}' added/updated.")
except Exception as e:
except Exception as e: # noqa: BLE001
printer.error(str(e))
return
@@ -201,7 +202,7 @@ class AIHandler:
try:
self.app.services.ai.configure_mcp(name, remove=True)
printer.success(f"MCP server '{name}' removed.")
except Exception as e:
except Exception as e: # noqa: BLE001
printer.error(str(e))
return
@@ -214,13 +215,54 @@ class AIHandler:
try:
self.app.services.ai.configure_mcp(name, enabled=enabled)
printer.success(f"MCP server '{name}' {'enabled' if enabled else 'disabled'}.")
except Exception as e:
except Exception as e: # noqa: BLE001
printer.error(str(e))
return
elif action == "serve":
transport = "stdio"
host = "127.0.0.1"
port = 8000
trust = getattr(args, "trust", False)
idx = 1
while idx < len(mcp_args):
tok = mcp_args[idx]
if tok in ["--sse", "sse"]:
transport = "sse"
elif tok in ["--stdio", "stdio"]:
transport = "stdio"
elif tok in ["--trust", "trust"]:
trust = True
elif tok in ["--port", "-p"]:
idx += 1
if idx < len(mcp_args):
try:
port = int(mcp_args[idx])
except ValueError:
printer.error(f"Invalid port: {mcp_args[idx]}")
return
elif tok.startswith("--port="):
try:
port = int(tok.split("=", 1)[1])
except ValueError:
printer.error(f"Invalid port: {tok}")
return
elif tok in ["--host", "-h"]:
idx += 1
if idx < len(mcp_args):
host = mcp_args[idx]
elif tok.startswith("--host="):
host = tok.split("=", 1)[1]
idx += 1
from ..mcp_server import ConnpyMCPServer
server = ConnpyMCPServer(config=self.app.config, services=self.app.services, trust=trust)
return server.run(transport=transport, host=host, port=port)
else:
printer.error(f"Unknown MCP action: {action}")
printer.info("Available actions: list, add, remove, enable, disable")
printer.info("Available actions: list, add, remove, enable, disable, serve")
return
# 2. Interactive Wizard Mode (if no arguments provided)
@@ -262,15 +304,16 @@ class AIHandler:
self.app.services.ai.configure_mcp(result["name"], remove=True)
printer.success(f"MCP server '{result['name']}' removed.")
except Exception as e:
except Exception as e: # noqa: BLE001
printer.error(str(e))
def _parse_auth_value(self, value):
if not value or value.lower() in ["none", "clear"]:
return None
import os
import yaml
import json
import os
import yaml
if os.path.exists(value):
try:
with open(value, "r") as f:
@@ -279,7 +322,7 @@ class AIHandler:
return json.loads(content)
except ValueError:
return yaml.safe_load(content)
except Exception as e:
except Exception as e: # noqa: BLE001
printer.error(f"Failed to read/parse auth file '{value}': {e}")
sys.exit(1)
@@ -291,6 +334,6 @@ class AIHandler:
if isinstance(parsed, dict):
return parsed
raise ValueError()
except Exception:
except Exception: # noqa: BLE001
printer.error("Auth parameter must be a valid JSON/YAML string, or a path to a JSON/YAML file.")
sys.exit(1)
+83 -1
View File
@@ -1,9 +1,22 @@
import re
import sys
import yaml
from .. import printer
from ..auth_utils import (
MAX_AUTH_PROMPTS_CAP,
MAX_HOP_HOST_ATTEMPTS_CAP,
MAX_TOTAL_AUTH_ATTEMPTS_CAP,
MIN_AUTH_PROMPTS,
MIN_HOP_HOST_ATTEMPTS,
MIN_TOTAL_AUTH_ATTEMPTS,
validate_strict_int_range,
)
from ..services.exceptions import ConnpyError, InvalidConfigurationError
from .help_text import get_instructions
class ConfigHandler:
def __init__(self, app):
self.app = app
@@ -29,7 +42,12 @@ class ConfigHandler:
"sync_remote": self.set_sync_remote,
"shell_command": self.set_shell_config,
"shell_prompt": self.set_shell_config,
"shell_os": self.set_shell_config
"shell_os": self.set_shell_config,
"max_auth_prompts": self.set_auth_budget,
"max_total_auth_attempts": self.set_auth_budget,
"max_hop_host_attempts": self.set_auth_budget,
"max_global_user_attempts": self.set_auth_budget,
"disable_fallback_on_insecure_hostkey": self.set_disable_fallback_on_insecure_hostkey,
}
handler = actions.get(getattr(args, "command", None))
if handler:
@@ -202,3 +220,67 @@ class ConfigHandler:
except (ConnpyError, InvalidConfigurationError) as e:
printer.error(str(e))
def set_auth_budget(self, args):
key = args.command
raw_val = args.data[0] if isinstance(args.data, list) else args.data
bounds_map = {
"max_auth_prompts": (MIN_AUTH_PROMPTS, MAX_AUTH_PROMPTS_CAP),
"max_total_auth_attempts": (MIN_TOTAL_AUTH_ATTEMPTS, MAX_TOTAL_AUTH_ATTEMPTS_CAP),
"max_hop_host_attempts": (MIN_HOP_HOST_ATTEMPTS, MAX_HOP_HOST_ATTEMPTS_CAP),
"max_global_user_attempts": (1, 1000),
}
if key not in bounds_map:
raise InvalidConfigurationError(f"Unknown auth budget setting: {key}")
if isinstance(raw_val, bool):
msg = f"Invalid {key}: boolean is not permitted"
printer.error(msg)
raise InvalidConfigurationError(msg)
if isinstance(raw_val, int):
val = raw_val
elif isinstance(raw_val, str) and re.fullmatch(r"[+-]?\d+", raw_val.strip()):
val = int(raw_val.strip())
else:
msg = f"Invalid {key}: expected integer, got {raw_val!r}"
printer.error(msg)
raise InvalidConfigurationError(msg)
if val == 0:
cfg_obj = self.app.config
if not hasattr(cfg_obj, "assert_called") and callable(
getattr(cfg_obj, "transactional_update", None)
):
cfg_obj.transactional_update(lambda c: c.config.pop(key, None))
else:
cfg_obj.config.pop(key, None)
cfg_obj._saveconfig(cfg_obj.file)
printer.success("Config saved")
return
min_v, max_v = bounds_map[key]
try:
validate_strict_int_range(val, key, min_v, max_v)
except ValueError as exc:
printer.error(str(exc))
raise InvalidConfigurationError(str(exc)) from None
self.app.services.config_svc.update_setting(key, val)
printer.success("Config saved")
def set_disable_fallback_on_insecure_hostkey(self, args):
raw_val = args.data[0] if isinstance(args.data, list) else args.data
if isinstance(raw_val, bool):
val = raw_val
elif isinstance(raw_val, str) and raw_val.strip().lower() in ("true", "false"):
val = raw_val.strip().lower() == "true"
else:
msg = "Invalid disable_fallback_on_insecure_hostkey: expected 'true' or 'false'"
printer.error(msg)
raise InvalidConfigurationError(msg)
self.app.services.config_svc.update_setting(
"disable_fallback_on_insecure_hostkey",
val,
)
printer.success("Config saved")
+132 -15
View File
@@ -1,6 +1,77 @@
import ast
import json
from ..auth_utils import (
MAX_FALLBACK_ENTRIES,
MAX_JSON_INPUT_BYTES,
MAX_JSON_INPUT_DEPTH,
validate_auth_budget_tags,
validate_fallback_entries,
validate_json_depth_and_size,
)
from .validators import Validators
def _parse_tags_input(raw_tags):
if raw_tags in ("", None):
return ""
if isinstance(raw_tags, dict):
validate_json_depth_and_size(
raw_tags,
max_bytes=MAX_JSON_INPUT_BYTES,
max_depth=MAX_JSON_INPUT_DEPTH,
)
validate_auth_budget_tags(raw_tags)
return raw_tags
s = str(raw_tags)
if s.startswith("@"):
return s
if len(s.encode("utf-8", errors="replace")) > MAX_JSON_INPUT_BYTES:
raise ValueError(f"Tags exceed maximum size of {MAX_JSON_INPUT_BYTES} bytes")
try:
parsed = json.loads(s)
except (ValueError, TypeError):
parsed = ast.literal_eval(s)
validate_json_depth_and_size(
parsed,
max_bytes=MAX_JSON_INPUT_BYTES,
max_depth=MAX_JSON_INPUT_DEPTH,
)
validate_auth_budget_tags(parsed)
return parsed
def _parse_fallback_input(raw_fb):
if raw_fb in (None, "", []):
return ""
if isinstance(raw_fb, str):
stripped = raw_fb.strip()
if not stripped:
return ""
if stripped.startswith(("[", "{")):
parsed = validate_json_depth_and_size(
stripped,
max_bytes=MAX_JSON_INPUT_BYTES,
max_depth=MAX_JSON_INPUT_DEPTH,
)
return validate_fallback_entries(parsed, max_entries=MAX_FALLBACK_ENTRIES)
validated = validate_fallback_entries(stripped, max_entries=MAX_FALLBACK_ENTRIES)
if (
stripped.startswith("@")
and not stripped.startswith("@@")
and "," in stripped
and not stripped.startswith("compose:")
):
return validated
return stripped
validate_json_depth_and_size(
raw_fb,
max_bytes=MAX_JSON_INPUT_BYTES,
max_depth=MAX_JSON_INPUT_DEPTH,
)
return validate_fallback_entries(raw_fb, max_entries=MAX_FALLBACK_ENTRIES)
class Forms:
def __init__(self, app):
self.app = app
@@ -18,6 +89,7 @@ class Forms:
questions.append(inquirer.Confirm("jumphost", message="Edit jumphost?"))
questions.append(inquirer.Confirm("user", message="Edit User?"))
questions.append(inquirer.Confirm("password", message="Edit password?"))
questions.append(inquirer.Confirm("fallback", message="Edit fallback credentials?"))
return inquirer.prompt(questions)
def questions_nodes(self, unique, uniques=None, edit=None):
@@ -28,11 +100,13 @@ class Forms:
defaults["tags"] = ""
if "jumphost" not in defaults:
defaults["jumphost"] = ""
except Exception:
defaults = {"host": "", "protocol": "", "port": "", "user": "", "options": "", "logs": "", "tags": "", "password": "", "jumphost": ""}
if "fallback" not in defaults:
defaults["fallback"] = ""
except Exception: # noqa: BLE001
defaults = {"host": "", "protocol": "", "port": "", "user": "", "options": "", "logs": "", "tags": "", "password": "", "jumphost": "", "fallback": ""}
node = {}
if edit is None:
edit = {"host": True, "protocol": True, "port": True, "user": True, "password": True, "options": True, "logs": True, "tags": True, "jumphost": True}
edit = {"host": True, "protocol": True, "port": True, "user": True, "password": True, "options": True, "logs": True, "tags": True, "jumphost": True, "fallback": True}
questions = []
if edit["host"]:
questions.append(inquirer.Text("host", message="Add Hostname or IP", validate=self.validators.host_validation, default=defaults["host"]))
@@ -70,6 +144,8 @@ class Forms:
questions.append(inquirer.List("password", message="Password: Use a local password, no password or a list of profiles to reference?", choices=["Local Password", "Profiles", "No Password"]))
else:
node["password"] = defaults["password"]
if not edit.get("fallback"):
node["fallback"] = defaults.get("fallback", "")
answer = inquirer.prompt(questions)
if answer is None:
@@ -87,12 +163,28 @@ class Forms:
passa = inquirer.prompt(passq)
if passa is None:
return False
answer["password"] = passa["password"].split(",")
answer["password"] = [p.strip() for p in passa["password"].split(",") if p.strip()]
elif answer["password"] == "No Password":
answer["password"] = ""
if "tags" in answer and not answer["tags"].startswith("@") and answer["tags"]:
answer["tags"] = ast.literal_eval(answer["tags"])
if edit.get("fallback"):
raw_def_fb = defaults.get("fallback", "")
def_fb_str = json.dumps(raw_def_fb) if isinstance(raw_def_fb, (list, dict)) else str(raw_def_fb or "")
fbq = [
inquirer.Text(
"fallback",
message="Add fallback credentials (@profile, @p1,@p2, JSON list/dict, or compose:)",
validate=self.validators.fallback_validation,
default=def_fb_str.replace("{", "{{").replace("}", "}}"),
)
]
fba = inquirer.prompt(fbq)
if fba is None:
return False
answer["fallback"] = _parse_fallback_input(fba["fallback"])
if "tags" in answer and answer["tags"] and not str(answer["tags"]).startswith("@"):
answer["tags"] = _parse_tags_input(answer["tags"])
result = {**uniques, **answer, **node}
result["type"] = "connection"
@@ -106,11 +198,13 @@ class Forms:
defaults["tags"] = ""
if "jumphost" not in defaults:
defaults["jumphost"] = ""
except Exception:
defaults = {"host": "", "protocol": "", "port": "", "user": "", "options": "", "logs": "", "tags": "", "jumphost": ""}
if "fallback" not in defaults:
defaults["fallback"] = ""
except Exception: # noqa: BLE001
defaults = {"host": "", "protocol": "", "port": "", "user": "", "options": "", "logs": "", "tags": "", "jumphost": "", "fallback": ""}
profile = {}
if edit is None:
edit = {"host": True, "protocol": True, "port": True, "user": True, "password": True, "options": True, "logs": True, "tags": True, "jumphost": True}
edit = {"host": True, "protocol": True, "port": True, "user": True, "password": True, "options": True, "logs": True, "tags": True, "jumphost": True, "fallback": True}
questions = []
if edit["host"]:
questions.append(inquirer.Text("host", message="Add Hostname or IP", default=defaults["host"]))
@@ -148,17 +242,24 @@ class Forms:
questions.append(inquirer.Password("password", message="Set Password"))
else:
profile["password"] = defaults["password"]
if edit.get("fallback"):
raw_def_fb = defaults.get("fallback", "")
def_fb_str = json.dumps(raw_def_fb) if isinstance(raw_def_fb, (list, dict)) else str(raw_def_fb or "")
questions.append(inquirer.Text("fallback", message="Add fallback credentials (@profile, @p1,@p2, JSON list/dict, or compose:)", validate=self.validators.fallback_validation, default=def_fb_str.replace("{", "{{").replace("}", "}}")))
else:
profile["fallback"] = defaults.get("fallback", "")
answer = inquirer.prompt(questions)
if answer is None:
return False
if "password" in answer:
if answer["password"] != "":
if "password" in answer and answer["password"] != "":
answer["password"] = self.app.services.config_svc.encrypt_password(answer["password"])
if "tags" in answer and answer["tags"]:
answer["tags"] = ast.literal_eval(answer["tags"])
answer["tags"] = _parse_tags_input(answer["tags"])
if "fallback" in answer:
answer["fallback"] = _parse_fallback_input(answer["fallback"])
result = {**answer, **profile}
result["id"] = unique
@@ -187,17 +288,33 @@ class Forms:
if answer["password"] == "Local Password":
passq = [inquirer.Password("password", message="Set Password")]
passa = inquirer.prompt(passq)
if passa is None:
return False
answer["password"] = self.app.services.config_svc.encrypt_password(passa["password"])
elif answer["password"] == "Profiles":
passq = [(inquirer.Text("password", message="Set a @profile or a comma separated list of @profiles", validate=self.validators.pass_validation))]
passa = inquirer.prompt(passq)
answer["password"] = passa["password"].split(",")
if passa is None:
return False
answer["password"] = [p.strip() for p in passa["password"].split(",") if p.strip()]
elif answer["password"] == "No Password":
answer["password"] = ""
fbq = [
inquirer.Text(
"fallback",
message="Add fallback credentials (@profile, @p1,@p2, JSON list/dict, or compose:)",
validate=self.validators.fallback_validation,
)
]
fba = inquirer.prompt(fbq)
if fba is None:
return False
answer["fallback"] = _parse_fallback_input(fba["fallback"])
answer["type"] = "connection"
if "tags" in answer and not answer["tags"].startswith("@") and answer["tags"]:
answer["tags"] = ast.literal_eval(answer["tags"])
if "tags" in answer and answer["tags"] and not str(answer["tags"]).startswith("@"):
answer["tags"] = _parse_tags_input(answer["tags"])
return answer
+1
View File
@@ -144,6 +144,7 @@ class NodeHandler:
uniques = self.app.services.nodes.explode_unique(args.data)
if not uniques:
raise InvalidConfigurationError(f"Invalid folder {args.data}")
self.app.services.nodes.validate_parent_folder(args.data, is_folder=True)
self.app.services.nodes.add_node(args.data, {}, is_folder=True)
printer.success(f"{args.data} added successfully")
else:
+4 -1
View File
@@ -162,7 +162,10 @@ class RunHandler:
path = args.data[0]
try:
with open(path, "r") as f:
playbook = yaml.load(f, Loader=yaml.FullLoader)
playbook = yaml.safe_load(f)
if not isinstance(playbook, dict):
raise TypeError("Invalid playbook format: expected a dictionary")
# Check preflight first before any task runs
if getattr(args, "preflight_ai", False):
+143 -26
View File
@@ -1,5 +1,19 @@
import re
import ast
import json
import re
from ..auth_utils import (
MAX_FALLBACK_ENTRIES,
MAX_JSON_INPUT_BYTES,
MAX_JSON_INPUT_DEPTH,
SanitizedAuthError,
is_compose_expression,
validate_auth_budget_tags,
validate_compose_syntax,
validate_fallback_entries,
validate_json_depth_and_size,
)
def _raise_val_err(reason):
import inquirer
@@ -56,50 +70,119 @@ class Validators:
return True
def pass_validation(self, answers, current, regex = "(^@.+$)"):
if is_compose_expression(current):
try:
validate_compose_syntax(current)
except SanitizedAuthError as exc:
_raise_val_err(str(exc))
return True
profiles = current.split(",")
for i in profiles:
if not re.match(regex, i) or i[1:] not in self.app.profiles:
_raise_val_err("Profile {} don't exist".format(i))
return True
def _validate_tags_dict(self, current: str):
if len(current.encode("utf-8", errors="replace")) > MAX_JSON_INPUT_BYTES:
_raise_val_err(f"Tags exceed maximum size of {MAX_JSON_INPUT_BYTES} bytes")
parsed = None
try:
parsed = json.loads(current)
except (ValueError, TypeError):
try:
parsed = ast.literal_eval(current)
except (ValueError, TypeError, SyntaxError, MemoryError, RecursionError):
parsed = None
if not isinstance(parsed, dict):
_raise_val_err("Tags should be a python dictionary.")
try:
validate_json_depth_and_size(
parsed,
max_bytes=MAX_JSON_INPUT_BYTES,
max_depth=MAX_JSON_INPUT_DEPTH,
)
validate_auth_budget_tags(parsed)
except (ValueError, TypeError, SanitizedAuthError) as exc:
_raise_val_err(str(exc))
return True
def tags_validation(self, answers, current):
if current.startswith("@"):
if current[1:] not in self.app.profiles:
_raise_val_err("Profile {} don't exist".format(current))
elif current != "":
isdict = False
try:
isdict = ast.literal_eval(current)
except Exception:
pass
if not isinstance (isdict, dict):
_raise_val_err("Tags should be a python dictionary.".format(current))
return self._validate_tags_dict(current)
return True
def profile_tags_validation(self, answers, current):
if current != "":
isdict = False
try:
isdict = ast.literal_eval(current)
except Exception:
pass
if not isinstance (isdict, dict):
_raise_val_err("Tags should be a python dictionary.".format(current))
return self._validate_tags_dict(current)
return True
def fallback_validation(self, answers, current):
if current in (None, ""):
return True
stripped = str(current).strip()
if not stripped:
return True
if len(stripped.encode("utf-8", errors="replace")) > MAX_JSON_INPUT_BYTES:
_raise_val_err(f"Fallback exceeds maximum size of {MAX_JSON_INPUT_BYTES} bytes")
if stripped.startswith("@") and not stripped.startswith("@@") and "+" not in stripped:
parts = [p.strip() for p in stripped.split(",")]
if any(not p for p in parts):
_raise_val_err("Fallback profile entry cannot be empty")
if len(parts) > MAX_FALLBACK_ENTRIES:
_raise_val_err(f"Fallback exceeds maximum of {MAX_FALLBACK_ENTRIES} entries")
for p in parts:
if not p.startswith("@") or not re.fullmatch(r"[A-Za-z0-9_.$:/-]+", p[1:]):
_raise_val_err(
"Comma-separated fallback entries must be valid @profile references"
)
if p[1:] not in self.app.profiles:
_raise_val_err(f"Profile {p} don't exist")
return True
try:
if is_compose_expression(stripped):
validate_compose_syntax(stripped)
return True
if stripped.startswith(("[", "{")):
validate_json_depth_and_size(
stripped,
max_bytes=MAX_JSON_INPUT_BYTES,
max_depth=MAX_JSON_INPUT_DEPTH,
)
validate_fallback_entries(stripped, max_entries=MAX_FALLBACK_ENTRIES)
except (ValueError, TypeError, SanitizedAuthError) as exc:
_raise_val_err(str(exc))
return True
def _get_all_valid_nodes(self) -> list[str]:
valid_nodes = list(getattr(self.app, "nodes_list", []) or [])
if hasattr(self.app, "services") and hasattr(self.app.services, "nodes"):
raw_fn = getattr(self.app.services.nodes, "_get_raw_nodes", None)
if callable(raw_fn) and not hasattr(raw_fn, "assert_called"):
raw_res = raw_fn()
if isinstance(raw_res, (list, tuple, set)):
valid_nodes = list(dict.fromkeys([*raw_res, *valid_nodes]))
return valid_nodes
def _node_exists(self, target: str) -> bool:
valid_nodes = self._get_all_valid_nodes()
if getattr(self.app, "case", True):
return target in valid_nodes
return target.lower() in {n.lower() for n in valid_nodes}
def jumphost_validation(self, answers, current):
if current.startswith("@"):
if current[1:] not in self.app.profiles:
_raise_val_err("Profile {} don't exist".format(current))
elif current != "":
if current not in self.app.nodes_list:
_raise_val_err("Node {} don't exist.".format(current))
_raise_val_err(f"Profile {current} don't exist")
elif current != "" and not self._node_exists(current):
_raise_val_err(f"Node {current} don't exist.")
return True
def profile_jumphost_validation(self, answers, current):
if current != "":
if current not in self.app.nodes_list:
_raise_val_err("Node {} don't exist.".format(current))
if current != "" and not self._node_exists(current):
_raise_val_err(f"Node {current} don't exist.")
return True
def default_validation(self, answers, current):
@@ -117,12 +200,46 @@ class Validators:
return True
def bulk_folder_validation(self, answers, current):
if not self.app.case:
case = getattr(self.app, "case", False)
if not case:
current = current.lower()
candidate = current
if "/" in current:
candidate = current.split("/")[0]
candidate = current.split("/")[0] if "/" in current else current
if hasattr(self.app, "services") and hasattr(self.app.services, "context"):
raw_folders = list(getattr(self.app, "folders", []) or [])
if hasattr(self.app.services, "nodes"):
raw_fn = getattr(self.app.services.nodes, "_get_raw_folders", None)
if callable(raw_fn) and not hasattr(raw_fn, "assert_called"):
raw_res = raw_fn()
if isinstance(raw_res, (list, tuple, set)):
raw_folders = list(raw_res)
def _folder_exists_norm(folder_name: str) -> bool:
if case:
return folder_name in raw_folders
return folder_name.lower() in {f.lower() for f in raw_folders}
if current != "" and not _folder_exists_norm(candidate):
_raise_val_err(f"Location {current} don't exist")
ctx_svc = self.app.services.context
if ctx_svc is not None and not ctx_svc.is_default_all_context():
raw_ids = answers.get("ids", "") if isinstance(answers, dict) else ""
for nid in str(raw_ids).split(","):
nid_stripped = nid.strip()
if not nid_stripped:
continue
nid_norm = nid_stripped if case else nid_stripped.lower()
full_id = f"{nid_norm}{candidate}" if candidate else nid_norm
if not ctx_svc.matches_active_context(full_id):
_raise_val_err(
f"Node '{full_id}' does not match active context '{ctx_svc.current_context}'."
)
parent = full_id.partition("@")[2]
if parent and not _folder_exists_norm(f"@{parent}"):
_raise_val_err(f"Location @{parent} don't exist")
return True
matches = list(filter(lambda k: k == candidate, self.app.folders))
if current != "" and len(matches) == 0:
+47 -18
View File
@@ -1,5 +1,7 @@
import sys
#!/usr/bin/env python3
import os
import sys
def load_txt_cache(filepath):
try:
@@ -27,7 +29,7 @@ def get_cwd(words, option=None, folderonly=False):
for i in range(len(pathstrings)):
if os.path.isdir(pathstrings[i]):
pathstrings[i] += '/'
pathstrings = [s[2:] if s.startswith('./') else s for s in pathstrings]
pathstrings = [s.removeprefix('./') for s in pathstrings]
if folderonly:
pathstrings = [s for s in pathstrings if os.path.isdir(s)]
return pathstrings
@@ -43,7 +45,7 @@ def _get_plugins(which, defaultdir):
try:
with open(pref_path) as f:
preferences = json.load(f)
except Exception:
except Exception: # noqa: BLE001
preferences = {}
# Load service mode
@@ -71,8 +73,8 @@ def _get_plugins(which, defaultdir):
# Get plugins from all directories
user_enabled, user_disabled, user_all_plugins = get_plugins_from_directory(defaultdir + "/plugins")
core_enabled, core_disabled, core_all_plugins = get_plugins_from_directory(core_path)
remote_enabled, remote_disabled, remote_all_plugins = get_plugins_from_directory(remote_path)
_core_enabled, _core_disabled, core_all_plugins = get_plugins_from_directory(core_path)
_remote_enabled, _remote_disabled, remote_all_plugins = get_plugins_from_directory(remote_path)
# Calculate final paths respecting priorities and preferences
# Priority: User Local > Core Local > Remote (unless preferred)
@@ -115,7 +117,7 @@ def _get_users(configdir):
data = yaml.safe_load(f) or {}
if isinstance(data, dict) and "users" in data:
return list(data["users"].keys())
except Exception:
except Exception: # noqa: BLE001, S110
pass
return []
@@ -135,7 +137,7 @@ def _get_sso_providers(configdir):
providers = sso.get("providers", {})
if isinstance(providers, dict):
return list(providers.keys())
except Exception:
except Exception: # noqa: BLE001, S110
pass
return []
@@ -210,12 +212,22 @@ def _build_tree(nodes, folders, profiles, plugins, configdir):
}
# State Machine Definitions
serve_dict = {"__exclude_used__": True}
serve_dict.update({
"--sse": serve_dict,
"--port": {"*": serve_dict},
"--trust": serve_dict,
"--help": None,
"-h": None
})
mcp_dict = {
"list": None,
"add": {"*": {"*": {"*": None}}}, # name url [os]
"remove": {"*": None},
"enable": {"*": None},
"disable": {"*": None},
"serve": serve_dict,
"--help": None, "-h": None
}
@@ -236,9 +248,26 @@ def _build_tree(nodes, folders, profiles, plugins, configdir):
"--fzf-wrapper": ["bash", "zsh"],
"--service-mode": ["local", "remote"],
"--sync-remote": ["true", "false"],
"--disable-fallback-on-insecure-hostkey": ["true", "false"],
"--help": None, "-h": None,
}
for opt in ["--keepalive", "--engineer-model", "--engineer-api-key", "--architect-model", "--architect-api-key", "--theme", "--remote", "--trusted-commands", "--shell-command", "--shell-prompt", "--shell-os"]:
for opt in [
"--keepalive",
"--engineer-model",
"--engineer-api-key",
"--architect-model",
"--architect-api-key",
"--theme",
"--remote",
"--trusted-commands",
"--shell-command",
"--shell-prompt",
"--shell-os",
"--max-auth-prompts",
"--max-total-auth-attempts",
"--max-hop-host-attempts",
"--max-global-user-attempts",
]:
config_dict[opt] = {"*": config_dict}
config_dict["--configfolder"] = {"__extra__": lambda w: get_cwd(w, "--configfolder", True), "*": config_dict}
config_dict["--engineer-auth"] = {"__extra__": lambda w: get_cwd(w, "--engineer-auth"), "*": config_dict}
@@ -291,7 +320,7 @@ def _build_tree(nodes, folders, profiles, plugins, configdir):
# Long flags are offered; short forms (-d/-t) only used for navigation.
# Two states: before node (offer nodes + remaining long flags)
# after node (offer only remaining long flags, no more nodes)
connect_flags_long = ["--debug", "--sftp"]
_connect_flags_long = ["--debug", "--sftp"]
connect_flags_all = ["--debug", "-d", "--sftp", "-t"]
# Post-node: only offer remaining long flags
@@ -443,7 +472,7 @@ def main():
try:
with open(pathfile, "r") as f:
configdir = f.read().strip()
except (FileNotFoundError, IOError):
except (OSError, FileNotFoundError):
configdir = defaultdir
cachefile = configdir + '/.config.cache.json'
@@ -477,8 +506,8 @@ def main():
module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module)
module.get_cwd = get_cwd
except Exception:
exit()
except Exception: # noqa: BLE001
sys.exit()
# New API: _connpy_tree → integrate into main tree and use resolver
if hasattr(module, "_connpy_tree"):
@@ -493,20 +522,20 @@ def main():
try:
with open(cachefile, "r") as jsonconf:
info["config"] = json.load(jsonconf)
except Exception:
except Exception: # noqa: BLE001
try:
import yaml
with open(configdir + '/config.yaml', "r") as yamlconf:
info["config"] = yaml.safe_load(yamlconf)
except Exception:
except Exception: # noqa: BLE001
info["config"] = {}
try:
plugin_completion = getattr(module, "_connpy_completion")
plugin_completion = module._connpy_completion
strings = plugin_completion(wordsnumber, words, info)
except Exception:
exit()
except Exception: # noqa: BLE001
sys.exit()
else:
exit()
sys.exit()
# --- Tree-based completion ---
else:
+698 -213
View File
File diff suppressed because it is too large. Load diff
+15 -18
View File
@@ -1,20 +1,20 @@
#!/usr/bin/env python3
#Imports
import argparse
import os
import re
import argparse
import sys
import yaml
import sys
from .core import node,nodes
from ._version import __version__
from . import printer
from .api import start_api,stop_api,debug_api
import yaml
from rich_argparse import RichHelpFormatter
from . import printer
from .api import debug_api, start_api, stop_api
from .cli.help_text import get_help
from .cli.helpers import folders_completer, nodes_completer
from .core import node, nodes
from .plugins import Plugins
from .services.exceptions import ConnpyError, ProfileNotFoundError, ReservedNameError
from rich_argparse import RichHelpFormatter
# Bridge rich-argparse with our design system
RichHelpFormatter.console = printer.console
RichHelpFormatter.styles.update({
@@ -28,14 +28,6 @@ RichHelpFormatter.styles.update({
})
RichHelpFormatter.group_name_formatter = str.upper
from .cli import (
NodeHandler, ProfileHandler, ConfigHandler, RunHandler,
AIHandler, APIHandler, PluginHandler, ImportExportHandler,
ContextHandler, SSOHandler
)
from .cli.helpers import nodes_completer, folders_completer, profiles_completer
from .cli.help_text import get_help
console = printer.console
#functions and classes
@@ -319,7 +311,7 @@ class connapp:
aiparser.add_argument("--session", nargs=1, help="Resume a specific AI session by ID")
aiparser.add_argument("--resume", action="store_true", help="Resume the most recent AI session")
aiparser.add_argument("--delete", "--delete-session", dest="delete_session", nargs=1, help="Delete an AI session by ID")
aiparser.add_argument("--mcp", nargs='*', metavar=('ACTION', 'NAME'), help="Manage MCP servers. Actions: list, add, remove, enable, disable. Leave empty for interactive wizard.")
aiparser.add_argument("--mcp", nargs=argparse.REMAINDER, metavar=('ACTION', 'NAME'), help="Manage MCP servers. Actions: list, add, remove, enable, disable, serve. Leave empty for interactive wizard.")
aiparser.set_defaults(func=self._ai.dispatch)
#RUNPARSER
runparser = subparsers.add_parser("run", help="Run scripts or commands on nodes", description="Run scripts or commands on nodes", formatter_class=RichHelpFormatter)
@@ -389,6 +381,11 @@ class connapp:
configcrud.add_argument("--shell-command", dest="shell_command", nargs=1, action=self._store_type, help="Set default shell command", metavar="COMMAND")
configcrud.add_argument("--shell-prompt", dest="shell_prompt", nargs=1, action=self._store_type, help="Set shell prompt regex for AI", metavar="REGEX")
configcrud.add_argument("--shell-os", dest="shell_os", nargs=1, action=self._store_type, help="Set shell OS hint for AI", metavar="OS")
configcrud.add_argument("--max-auth-prompts", dest="max_auth_prompts", nargs=1, action=self._store_type, help="Set max auth prompts per connection (1-5, 0 to reset)", type=int, metavar="INT")
configcrud.add_argument("--max-total-auth-attempts", dest="max_total_auth_attempts", nargs=1, action=self._store_type, help="Set max total auth attempts per host (1-10, 0 to reset)", type=int, metavar="INT")
configcrud.add_argument("--max-hop-host-attempts", dest="max_hop_host_attempts", nargs=1, action=self._store_type, help="Set max auth attempts per jumphost (1-10, 0 to reset)", type=int, metavar="INT")
configcrud.add_argument("--max-global-user-attempts", dest="max_global_user_attempts", nargs=1, action=self._store_type, help="Set max global auth attempts per user (1-1000, 0 to reset)", type=int, metavar="INT")
configcrud.add_argument("--disable-fallback-on-insecure-hostkey", dest="disable_fallback_on_insecure_hostkey", nargs=1, action=self._store_type, help="Disable AAA fallback when StrictHostKeyChecking=no", choices=["true", "false"])
configparser.add_argument("--trusted-commands", dest="trusted_commands", nargs=1, action=self._store_type, help="Set custom trusted commands regexes (comma separated)", metavar="REGEX,REGEX")
configparser.set_defaults(func=self._config.dispatch)
+2296 -330
View File
File diff suppressed because it is too large. Load diff
+96 -54
View File
@@ -1,26 +1,37 @@
import grpc
from concurrent import futures
from google.protobuf.empty_pb2 import Empty
import os
import ctypes
import threading
import contextvars
import datetime
import functools
import json
import os
from concurrent import futures
import grpc
from google.protobuf.empty_pb2 import Empty
# Suppress harmless but noisy gRPC fork() warnings from pexpect child processes
os.environ["GRPC_VERBOSITY"] = "NONE"
os.environ["GRPC_ENABLE_FORK_SUPPORT"] = "0"
from . import connpy_pb2, connpy_pb2_grpc, remote_plugin_pb2, remote_plugin_pb2_grpc
import json
from .utils import to_value, from_value, to_struct, from_struct
from ..services.exceptions import ConnpyError
from .. import printer
from ..auth_utils import (
MAX_FALLBACK_ENTRIES,
MAX_JSON_INPUT_BYTES,
MAX_JSON_INPUT_DEPTH,
SanitizedAuthError,
sanitize_exception,
validate_auth_budget_tags,
validate_fallback_entries,
validate_json_depth_and_size,
)
from ..services.exceptions import ConnpyError
from . import connpy_pb2, connpy_pb2_grpc, remote_plugin_pb2, remote_plugin_pb2_grpc
from .utils import from_struct, from_value, to_struct, to_value
_current_user = contextvars.ContextVar("current_user", default=None)
def handle_errors(func):
import inspect
if inspect.isgeneratorfunction(func):
@functools.wraps(func)
def wrapper(*args, **kwargs):
try:
for item in func(*args, **kwargs):
@@ -41,6 +52,7 @@ def handle_errors(func):
printer.clear_thread_state()
return wrapper
else:
@functools.wraps(func)
def wrapper(*args, **kwargs):
try:
return func(*args, **kwargs)
@@ -60,11 +72,19 @@ def handle_errors(func):
printer.clear_thread_state()
return wrapper
class NodeServicer(connpy_pb2_grpc.NodeServiceServicer):
def __init__(self, provider, registry=None, debug=False):
def _ensure_server_provider(provider):
if not hasattr(provider, "mode"):
from connpy.services.provider import ServiceProvider
provider = ServiceProvider(provider, mode="local")
return ServiceProvider(provider, mode="local", enforce_context=False)
if hasattr(provider, "set_enforce_context") and callable(provider.set_enforce_context):
provider.set_enforce_context(False)
return provider
class NodeServicer(connpy_pb2_grpc.NodeServiceServicer):
def __init__(self, provider, registry=None, debug=False):
provider = _ensure_server_provider(provider)
self._fallback_provider = provider
self._registry = registry
self.server_debug = debug
@@ -115,8 +135,24 @@ class NodeServicer(connpy_pb2_grpc.NodeServiceServicer):
self.server_console.print(f"[debug][DEBUG][/debug] gRPC interact_node request for: [bold cyan]{unique_id}[/bold cyan]")
if first_req.connection_params_json:
import json
params = json.loads(first_req.connection_params_json)
try:
params = validate_json_depth_and_size(
first_req.connection_params_json,
max_bytes=MAX_JSON_INPUT_BYTES,
max_depth=MAX_JSON_INPUT_DEPTH,
)
if not isinstance(params, dict):
raise TypeError("connection_params_json must be a JSON object")
if "fallback" in params and params["fallback"] not in (None, "", []):
validate_fallback_entries(
params["fallback"],
max_entries=MAX_FALLBACK_ENTRIES,
)
if "tags" in params and params["tags"] not in ("", None):
validate_auth_budget_tags(params["tags"])
except (ValueError, TypeError, SanitizedAuthError) as exc:
context.abort(grpc.StatusCode.INVALID_ARGUMENT, str(exc))
base_node_id = params.get("base_node")
# Valid attributes that a node object accepts
valid_attrs = ['host', 'options', 'logs', 'password', 'port', 'protocol', 'user', 'jumphost']
@@ -134,6 +170,8 @@ class NodeServicer(connpy_pb2_grpc.NodeServiceServicer):
for attr in valid_attrs:
if attr in params:
device[attr] = params[attr]
if "fallback" in params:
device["fallback"] = params["fallback"]
if "tags" in params:
device_tags = device.get("tags", {})
@@ -147,20 +185,30 @@ class NodeServicer(connpy_pb2_grpc.NodeServiceServicer):
else:
# base_node not found, fall back to dynamic
node_name = params.get("name", fallback_id)
n = node(node_name, host=params.get("host", ""), config=user_config)
for attr in valid_attrs:
if attr in params:
setattr(n, attr, params[attr])
dyn_kwargs = {k: params[k] for k in valid_attrs if k in params and k != "host"}
if "fallback" in params:
dyn_kwargs["fallback"] = params["fallback"]
if "tags" in params:
n.tags = params["tags"]
dyn_kwargs["tags"] = params["tags"]
n = node(
node_name,
host=params.get("host", ""),
config=user_config,
**dyn_kwargs,
)
else:
node_name = params.get("name", fallback_id)
n = node(node_name, host=params.get("host", ""), config=user_config)
for attr in valid_attrs:
if attr in params:
setattr(n, attr, params[attr])
dyn_kwargs = {k: params[k] for k in valid_attrs if k in params and k != "host"}
if "fallback" in params:
dyn_kwargs["fallback"] = params["fallback"]
if "tags" in params:
n.tags = params["tags"]
dyn_kwargs["tags"] = params["tags"]
n = node(
node_name,
host=params.get("host", ""),
config=user_config,
**dyn_kwargs,
)
n._orig_protocol = n.protocol
if sftp:
n.protocol = "sftp"
@@ -181,9 +229,11 @@ class NodeServicer(connpy_pb2_grpc.NodeServiceServicer):
debug_chunks = []
if debug:
from io import StringIO
from rich.console import Console as RichConsole
from ..printer import connpy_theme
from .. import printer as _printer
from ..printer import connpy_theme
def remote_logger(msg_type, message):
buf = StringIO()
@@ -204,7 +254,14 @@ class NodeServicer(connpy_pb2_grpc.NodeServiceServicer):
else:
remote_logger = None
connect = n._connect(debug=debug, logger=remote_logger)
try:
connect = n._connect(
debug=debug,
logger=remote_logger,
attempt_tracker=getattr(provider, "auth_tracker", None),
)
except Exception as exc: # noqa: BLE001
connect = str(sanitize_exception(exc, code="AUTH_FAILED"))
# Send debug output to client before checking result (always show the command)
for chunk in debug_chunks:
@@ -535,10 +592,9 @@ class NodeServicer(connpy_pb2_grpc.NodeServiceServicer):
continue
asyncio.run(n._async_interact_loop(remote_stream, resize_callback, copilot_handler=remote_copilot_handler))
except Exception as e:
import traceback
print(f"[ERROR in run_async_loop] {e}")
traceback.print_exc()
except Exception as e: # noqa: BLE001
sanitized_err = sanitize_exception(e, code="TRANSPORT_ERROR", context_label="interact stream")
print(f"[ERROR in run_async_loop] {sanitized_err}")
finally:
n._teardown_interact_environment()
response_queue.put(None) # Signal EOF
@@ -641,9 +697,7 @@ class NodeServicer(connpy_pb2_grpc.NodeServiceServicer):
class ProfileServicer(connpy_pb2_grpc.ProfileServiceServicer):
def __init__(self, provider, registry=None):
if not hasattr(provider, "mode"):
from connpy.services.provider import ServiceProvider
provider = ServiceProvider(provider, mode="local")
provider = _ensure_server_provider(provider)
self._fallback_provider = provider
self._registry = registry
@@ -695,9 +749,7 @@ class ProfileServicer(connpy_pb2_grpc.ProfileServiceServicer):
class ConfigServicer(connpy_pb2_grpc.ConfigServiceServicer):
def __init__(self, provider, registry=None):
if not hasattr(provider, "mode"):
from connpy.services.provider import ServiceProvider
provider = ServiceProvider(provider, mode="local")
provider = _ensure_server_provider(provider)
self._fallback_provider = provider
self._registry = registry
@@ -740,9 +792,7 @@ class ConfigServicer(connpy_pb2_grpc.ConfigServiceServicer):
class PluginServicer(connpy_pb2_grpc.PluginServiceServicer, remote_plugin_pb2_grpc.RemotePluginServiceServicer):
def __init__(self, provider, registry=None):
if not hasattr(provider, "mode"):
from connpy.services.provider import ServiceProvider
provider = ServiceProvider(provider, mode="local")
provider = _ensure_server_provider(provider)
self._fallback_provider = provider
self._registry = registry
@@ -798,9 +848,7 @@ class PluginServicer(connpy_pb2_grpc.PluginServiceServicer, remote_plugin_pb2_gr
class ExecutionServicer(connpy_pb2_grpc.ExecutionServiceServicer):
def __init__(self, provider, registry=None):
if not hasattr(provider, "mode"):
from connpy.services.provider import ServiceProvider
provider = ServiceProvider(provider, mode="local")
provider = _ensure_server_provider(provider)
self._fallback_provider = provider
self._registry = registry
@@ -929,9 +977,7 @@ class ExecutionServicer(connpy_pb2_grpc.ExecutionServiceServicer):
class ImportExportServicer(connpy_pb2_grpc.ImportExportServiceServicer):
def __init__(self, provider, registry=None):
if not hasattr(provider, "mode"):
from connpy.services.provider import ServiceProvider
provider = ServiceProvider(provider, mode="local")
provider = _ensure_server_provider(provider)
self._fallback_provider = provider
self._registry = registry
@@ -960,7 +1006,7 @@ class ImportExportServicer(connpy_pb2_grpc.ImportExportServiceServicer):
if request.value.startswith("---YAML---\n"):
import yaml
content = request.value[len("---YAML---\n"):]
data = yaml.load(content, Loader=yaml.FullLoader)
data = yaml.safe_load(content)
self.service.import_from_dict(data)
else:
self.service.import_from_file(request.value)
@@ -1064,9 +1110,7 @@ class StatusBridge:
class AIServicer(connpy_pb2_grpc.AIServiceServicer):
def __init__(self, provider, registry=None, debug=False):
if not hasattr(provider, "mode"):
from connpy.services.provider import ServiceProvider
provider = ServiceProvider(provider, mode="local")
provider = _ensure_server_provider(provider)
self._fallback_provider = provider
self._registry = registry
self.server_debug = debug
@@ -1351,9 +1395,7 @@ class AIServicer(connpy_pb2_grpc.AIServiceServicer):
class SystemServicer(connpy_pb2_grpc.SystemServiceServicer):
def __init__(self, provider, registry=None):
if not hasattr(provider, "mode"):
from connpy.services.provider import ServiceProvider
provider = ServiceProvider(provider, mode="local")
provider = _ensure_server_provider(provider)
self._fallback_provider = provider
self._registry = registry
@@ -1736,7 +1778,7 @@ def serve(config, port=8048, debug=False):
from connpy.grpc_layer.user_registry import UserRegistry
from connpy.services.provider import ServiceProvider
fallback_provider = ServiceProvider(config, mode="local")
fallback_provider = ServiceProvider(config, mode="local", enforce_context=False)
registry = UserRegistry(config.defaultdir)
# Check if trusted_gateway provider is configured if SSO Gateway Secret is present in env
+151 -21
View File
@@ -1,16 +1,19 @@
import grpc
import queue
import re
import threading
from functools import wraps
import grpc
from google.protobuf.empty_pb2 import Empty
from . import connpy_pb2, connpy_pb2_grpc, remote_plugin_pb2, remote_plugin_pb2_grpc
from .utils import to_value, from_value, to_struct, from_struct
from ..services.exceptions import ConnpyError
from ..hooks import MethodHook
from .. import printer
from ..cli.terminal_ui import CopilotInterface
from ..auth_utils import MAX_REMOTE_FILTER_NODES
from ..hooks import MethodHook
from ..services.exceptions import ConnpyError, NodeNotFoundError
from ..utils import log_cleaner
from . import connpy_pb2, connpy_pb2_grpc, remote_plugin_pb2, remote_plugin_pb2_grpc
from .utils import from_struct, from_value, to_struct, to_value
def handle_errors(func):
@wraps(func)
@@ -43,8 +46,48 @@ class NodeStub:
self.remote_host = remote_host
self.config = config
def _get_context_service(self):
if not getattr(self.config, "_enforce_context", True):
return None
if not isinstance(getattr(self.config, "config", None), dict):
return None
from ..services.context_service import ContextService
return ContextService(self.config)
def _ensure_in_active_context(
self, unique_id: str, *, is_folder: bool = False, not_found_msg: str | None = None
) -> None:
ctx_svc = self._get_context_service()
if (
ctx_svc is not None
and not ctx_svc.is_default_all_context()
and not ctx_svc.matches_active_context(unique_id)
):
if not_found_msg is not None:
raise NodeNotFoundError(not_found_msg)
label = "Folder" if is_folder else "Node"
raise NodeNotFoundError(f"{label} '{unique_id}' not found.")
@handle_errors
def _get_raw_nodes(self) -> list[str]:
req = connpy_pb2.FilterRequest(filter_str="", format_str="")
return from_value(self.stub.list_nodes(req).data) or []
@handle_errors
def _get_raw_folders(self) -> list[str]:
req = connpy_pb2.FilterRequest(filter_str="")
return from_value(self.stub.list_folders(req).data) or []
def _handle_remote_copilot(self, res, request_queue, response_queue, client_buffer_bytes, pause_generator, resume_generator, old_tty):
import json, asyncio, termios, sys, tty, queue, os
import asyncio
import json
import os
import sys
import termios
import tty
from ..cli.terminal_ui import CopilotInterface
from ..core import copilot_terminal_mode
from . import connpy_pb2
@@ -138,6 +181,7 @@ class NodeStub:
@handle_errors
def connect_node(self, unique_id, sftp=False, debug=False, logger=None):
self._ensure_in_active_context(unique_id, is_folder=False)
import sys
import select
import tty
@@ -429,16 +473,26 @@ class NodeStub:
@handle_errors
def list_nodes(self, filter_str=None, format_str=None):
req = connpy_pb2.FilterRequest(filter_str=filter_str or "", format_str=format_str or "")
return from_value(self.stub.list_nodes(req).data) or []
nodes = from_value(self.stub.list_nodes(req).data) or []
if not format_str:
ctx_svc = self._get_context_service()
if ctx_svc is not None and not ctx_svc.is_default_all_context():
nodes = ctx_svc.filter_node_list(result=nodes)
return nodes
@MethodHook
@handle_errors
def list_folders(self, filter_str=None):
req = connpy_pb2.FilterRequest(filter_str=filter_str or "")
return from_value(self.stub.list_folders(req).data) or []
folders = from_value(self.stub.list_folders(req).data) or []
ctx_svc = self._get_context_service()
if ctx_svc is not None and not ctx_svc.is_default_all_context():
folders = ctx_svc.filter_node_list(result=folders)
return folders
@handle_errors
def get_node_details(self, unique_id):
self._ensure_in_active_context(unique_id, is_folder=False)
return from_struct(self.stub.get_node_details(connpy_pb2.IdRequest(id=unique_id)).data)
@handle_errors
@@ -446,7 +500,11 @@ class NodeStub:
return from_value(self.stub.explode_unique(connpy_pb2.IdRequest(id=unique_id)).data)
@handle_errors
def validate_parent_folder(self, unique_id):
def validate_parent_folder(self, unique_id, is_folder=False):
ctx_svc = self._get_context_service()
if ctx_svc is not None:
ctx_svc.validate_in_active_context(unique_id, is_folder=is_folder)
if not is_folder:
self.stub.validate_parent_folder(connpy_pb2.IdRequest(id=unique_id))
@handle_errors
@@ -463,7 +521,7 @@ class NodeStub:
# node cache is what matters for fzf. We'll fetch profiles if we can.
# For now, let's sync what we have.
if nodes is not None or folders is not None or profiles is not None:
if self.config is not None and (nodes is not None or folders is not None or profiles is not None):
self.config._generate_nodes_cache(nodes=nodes, folders=folders, profiles=profiles)
def _trigger_local_cache_sync(self):
@@ -478,12 +536,16 @@ class NodeStub:
@handle_errors
def add_node(self, unique_id, data, is_folder=False):
ctx_svc = self._get_context_service()
if ctx_svc is not None:
ctx_svc.validate_in_active_context(unique_id, is_folder=is_folder)
req = connpy_pb2.NodeRequest(id=unique_id, data=to_struct(data), is_folder=is_folder)
self.stub.add_node(req)
self._trigger_local_cache_sync()
@handle_errors
def update_node(self, unique_id, data, save=True):
self._ensure_in_active_context(unique_id, is_folder=False)
req = connpy_pb2.NodeRequest(id=unique_id, data=to_struct(data), is_folder=False)
self.stub.update_node(req)
if save:
@@ -491,6 +553,7 @@ class NodeStub:
@handle_errors
def delete_node(self, unique_id, is_folder=False, save=True):
self._ensure_in_active_context(unique_id, is_folder=is_folder)
req = connpy_pb2.DeleteRequest(id=unique_id, is_folder=is_folder)
self.stub.delete_node(req)
if save:
@@ -498,12 +561,22 @@ class NodeStub:
@handle_errors
def move_node(self, src_id, dst_id, copy=False):
self._ensure_in_active_context(
src_id, is_folder=False, not_found_msg=f"Source node '{src_id}' not found."
)
ctx_svc = self._get_context_service()
if ctx_svc is not None:
ctx_svc.validate_in_active_context(dst_id, is_folder=False)
req = connpy_pb2.MoveRequest(src_id=src_id, dst_id=dst_id, copy=copy)
self.stub.move_node(req)
self._trigger_local_cache_sync()
@handle_errors
def bulk_add(self, ids, hosts, common_data):
ctx_svc = self._get_context_service()
if ctx_svc is not None:
for uid in ids:
ctx_svc.validate_in_active_context(uid, is_folder=False)
req = connpy_pb2.BulkRequest(ids=ids, hosts=hosts, common_data=to_struct(common_data))
self.stub.bulk_add(req)
self._trigger_local_cache_sync()
@@ -649,13 +722,58 @@ class PluginStub:
yield chunk.text
class ExecutionStub:
def __init__(self, channel, remote_host):
def __init__(self, channel, remote_host, config=None, node_stub=None):
self.stub = connpy_pb2_grpc.ExecutionServiceStub(channel)
self.remote_host = remote_host
self.config = config
self.node_stub = node_stub
def _get_context_service(self):
if not getattr(self.config, "_enforce_context", True):
return None
if not isinstance(getattr(self.config, "config", None), dict):
return None
from ..services.context_service import ContextService
return ContextService(self.config)
def _filter_candidates_by_filter_str(self, candidates: list[str], nodes_filter: str) -> list[str]:
if not nodes_filter:
return list(candidates)
cfg_dict = getattr(self.config, "config", None)
case = cfg_dict.get("case", False) if isinstance(cfg_dict, dict) else False
flags = 0 if case else re.IGNORECASE
return [c for c in candidates if re.search(nodes_filter, c, flags)]
def _resolve_remote_batches(self, nodes_filter):
ctx_svc = self._get_context_service()
if ctx_svc is not None and not ctx_svc.is_default_all_context():
if isinstance(nodes_filter, (list, tuple)):
permitted = ctx_svc.filter_node_list(result=list(nodes_filter))
else:
candidates = (
self.node_stub.list_nodes(filter_str=nodes_filter)
if self.node_stub is not None
else []
)
candidates = self._filter_candidates_by_filter_str(candidates, nodes_filter)
permitted = ctx_svc.filter_node_list(result=candidates)
if not permitted:
raise ConnpyError(f"No nodes found matching filter: {nodes_filter}")
patterns = [f"^{re.escape(n)}$" for n in permitted]
return [
patterns[i : i + MAX_REMOTE_FILTER_NODES]
for i in range(0, len(patterns), MAX_REMOTE_FILTER_NODES)
]
return [[nodes_filter] if isinstance(nodes_filter, str) else list(nodes_filter)]
@handle_errors
def run_commands(self, nodes_filter, commands, variables=None, parallel=10, timeout=10, folder=None, prompt=None, **kwargs):
nodes_list = [nodes_filter] if isinstance(nodes_filter, str) else list(nodes_filter)
batches = self._resolve_remote_batches(nodes_filter)
final_results = {}
on_complete = kwargs.get("on_node_complete")
for nodes_list in batches:
req = connpy_pb2.RunRequest(
nodes=nodes_list,
commands=commands,
@@ -668,9 +786,6 @@ class ExecutionStub:
if variables is not None:
req.vars.CopyFrom(to_struct(variables))
final_results = {}
on_complete = kwargs.get("on_node_complete")
for response in self.stub.run_commands(req):
if on_complete:
on_complete(response.unique_id, response.output, response.status)
@@ -683,7 +798,11 @@ class ExecutionStub:
@handle_errors
def test_commands(self, nodes_filter, commands, expected, variables=None, parallel=10, timeout=10, prompt=None, **kwargs):
nodes_list = [nodes_filter] if isinstance(nodes_filter, str) else list(nodes_filter)
batches = self._resolve_remote_batches(nodes_filter)
final_results = {}
on_complete = kwargs.get("on_node_complete")
for nodes_list in batches:
req = connpy_pb2.TestRequest(
nodes=nodes_list,
commands=commands,
@@ -697,9 +816,6 @@ class ExecutionStub:
if variables is not None:
req.vars.CopyFrom(to_struct(variables))
final_results = {}
on_complete = kwargs.get("on_node_complete")
for response in self.stub.test_commands(req):
result_dict = from_struct(response.test_result) if response.HasField("test_result") else {}
if on_complete:
@@ -710,7 +826,21 @@ class ExecutionStub:
@handle_errors
def run_cli_script(self, nodes_filter, script_path, parallel=10):
req = connpy_pb2.ScriptRequest(param1=nodes_filter, param2=script_path, parallel=parallel)
ctx_svc = self._get_context_service()
effective_filter = nodes_filter
if ctx_svc is not None and not ctx_svc.is_default_all_context():
candidates = (
self.node_stub.list_nodes(filter_str=nodes_filter)
if self.node_stub is not None
else []
)
if isinstance(nodes_filter, str):
candidates = self._filter_candidates_by_filter_str(candidates, nodes_filter)
permitted = ctx_svc.filter_node_list(result=candidates)
if not permitted:
raise ConnpyError(f"No nodes found matching filter: {nodes_filter}")
effective_filter = "|".join(f"^{re.escape(n)}$" for n in permitted)
req = connpy_pb2.ScriptRequest(param1=effective_filter, param2=script_path, parallel=parallel)
return from_struct(self.stub.run_cli_script(req).data)
class ImportExportStub:
+1 -1
View File
@@ -62,7 +62,7 @@ class UserRegistry:
try:
# Attempt a fresh configuration load
config = configfile(conf=conf_file, shared_config=self._shared_config)
new_provider = ServiceProvider(config, mode="local")
new_provider = ServiceProvider(config, mode="local", enforce_context=False)
# Successfully loaded, clean up the old provider
if old_provider:
+91 -27
View File
@@ -1,6 +1,7 @@
#!/usr/bin/env python3
#Imports
from functools import wraps, partial, update_wrapper
from functools import update_wrapper, wraps
from . import printer
#functions and classes
@@ -10,54 +11,118 @@ class MethodHook:
def __init__(self, func):
self.func = func
self.pre_hooks = [] # List to store registered pre-hooks
self.post_hooks = [] # List to store registered post-hooks
self.pre_hooks = [] # List to store registered class-level pre-hooks
self.post_hooks = [] # List to store registered class-level post-hooks
wraps(func)(self)
def register_pre_hook(self, hook):
"""Register a function to be called before the original function"""
if hook not in self.pre_hooks:
self.pre_hooks.append(hook)
def register_post_hook(self, hook):
"""Register a function to be called after the original function"""
if hook not in self.post_hooks:
self.post_hooks.append(hook)
def __call__(self, *args, **kwargs):
# Execute pre-hooks before the original function
for hook in self.pre_hooks:
# Fallback when called directly on descriptor or standalone function
inst_pre = []
inst_post = []
if args and hasattr(args[0], "__dict__"):
inst_dict = args[0].__dict__
inst_pre = inst_dict.get("_method_pre_hooks", {}).get(self.func.__name__, []) if isinstance(inst_dict.get("_method_pre_hooks"), dict) else []
inst_post = inst_dict.get("_method_post_hooks", {}).get(self.func.__name__, []) if isinstance(inst_dict.get("_method_post_hooks"), dict) else []
all_pre = list(self.pre_hooks) + [h for h in inst_pre if h not in self.pre_hooks]
for hook in all_pre:
try:
args, kwargs = hook(*args, **kwargs)
except Exception as e:
except Exception as e: # noqa: BLE001
hook_name = getattr(hook, "__name__", str(hook))
printer.error(f"{self.func.__name__} Pre-hook {hook_name} raised an exception: {e}")
result = self.func(*args, **kwargs)
# Execute post-hooks after the original function
if self.post_hooks:
#printer.info(f"Executing {len(self.post_hooks)} post-hooks for {self.func.__name__}...")
pass
for hook in self.post_hooks:
all_post = list(self.post_hooks) + [h for h in inst_post if h not in self.post_hooks]
for hook in all_post:
try:
result = hook(*args, **kwargs, result=result) # Pass result to hooks
except Exception as e:
result = hook(*args, **kwargs, result=result)
except Exception as e: # noqa: BLE001
hook_name = getattr(hook, "__name__", str(hook))
printer.error(f"{self.func.__name__} Post-hook {hook_name} raised an exception: {e}")
return result
def __get__(self, instance, owner):
if not instance:
if instance is None:
return self
else:
return self.make_callable(instance)
def make_callable(self, instance):
# Returns a callable that also allows access to hook registration methods
callable_instance = partial(self.__call__, instance)
callable_instance.register_pre_hook = self.register_pre_hook
callable_instance.register_post_hook = self.register_post_hook
return callable_instance
# Returns a callable bound to instance with per-instance hook registration
func_name = self.func.__name__
def register_pre_hook(self, hook):
"""Register a function to be called before the original function"""
self.pre_hooks.append(hook)
def bound_method(*args, **kwargs):
# Pre-hooks: class-level followed by instance-level
if hasattr(instance, "__dict__"):
inst_pre = instance.__dict__.get("_method_pre_hooks", {}).get(func_name, [])
inst_post = instance.__dict__.get("_method_post_hooks", {}).get(func_name, [])
else:
inst_pre = instance._method_pre_hooks.get(func_name, []) if hasattr(instance, "_method_pre_hooks") else []
inst_post = instance._method_post_hooks.get(func_name, []) if hasattr(instance, "_method_post_hooks") else []
all_pre = list(self.pre_hooks) + [h for h in inst_pre if h not in self.pre_hooks]
call_args = (instance,) + args
for hook in all_pre:
try:
call_args, kwargs = hook(*call_args, **kwargs)
except Exception as e: # noqa: BLE001
hook_name = getattr(hook, "__name__", str(hook))
printer.error(f"{func_name} Pre-hook {hook_name} raised an exception: {e}")
result = self.func(*call_args, **kwargs)
all_post = list(self.post_hooks) + [h for h in inst_post if h not in self.post_hooks]
for hook in all_post:
try:
result = hook(*call_args, **kwargs, result=result)
except Exception as e: # noqa: BLE001
hook_name = getattr(hook, "__name__", str(hook))
printer.error(f"{func_name} Post-hook {hook_name} raised an exception: {e}")
return result
wraps(self.func)(bound_method)
def register_pre_hook(hook):
if hasattr(instance, "__dict__"):
hooks = instance.__dict__.setdefault("_method_pre_hooks", {}).setdefault(func_name, [])
else:
if not hasattr(instance, "_method_pre_hooks"):
instance._method_pre_hooks = {}
hooks = instance._method_pre_hooks.setdefault(func_name, [])
if hook not in hooks:
hooks.append(hook)
def register_post_hook(hook):
if hasattr(instance, "__dict__"):
hooks = instance.__dict__.setdefault("_method_post_hooks", {}).setdefault(func_name, [])
else:
if not hasattr(instance, "_method_post_hooks"):
instance._method_post_hooks = {}
hooks = instance._method_post_hooks.setdefault(func_name, [])
if hook not in hooks:
hooks.append(hook)
bound_method.register_pre_hook = register_pre_hook
bound_method.register_post_hook = register_post_hook
bound_method.__self__ = instance
bound_method.__func__ = self.func
return bound_method
def register_post_hook(self, hook):
"""Register a function to be called after the original function"""
self.post_hooks.append(hook)
class ClassHook:
"""Decorator class to enable Class Modifying"""
@@ -91,4 +156,3 @@ class ClassHook:
"""Modify this specific instance."""
modification_func(instance)
return modify_instance
+98 -41
View File
@@ -1,9 +1,9 @@
import asyncio
import json
import logging
import os
import threading
from typing import Any, Dict, List, Optional
import logging
from contextlib import suppress
from typing import Any, ClassVar
try:
from mcp import ClientSession
@@ -17,32 +17,54 @@ logging.getLogger("mcp").setLevel(logging.CRITICAL)
logging.getLogger("httpx").setLevel(logging.CRITICAL)
logging.getLogger("httpcore").setLevel(logging.CRITICAL)
class MCPClientManager:
"""Manages MCP SSE client connections for connpy."""
_instance = None
_instances: ClassVar[dict[Any, "MCPClientManager"]] = {}
_instance: ClassVar["MCPClientManager | None"] = None
_lock = threading.Lock()
def __new__(cls, *args, **kwargs):
@classmethod
def _resolve_scope_key(cls, config=None, *args, **kwargs) -> Any:
cfg = config
if cfg is None and "config" in kwargs:
cfg = kwargs["config"]
if cfg is not None:
defaultdir = getattr(cfg, "defaultdir", None)
if defaultdir:
return os.path.abspath(str(defaultdir))
return id(cfg)
return "default"
def __new__(cls, config=None, *args, **kwargs):
key = cls._resolve_scope_key(config, *args, **kwargs)
with cls._lock:
if cls._instance is None:
cls._instance = super(MCPClientManager, cls).__new__(cls)
cls._instance._initialized = False
return cls._instance
if cls._instance is None and cls._instances:
cls._instances.clear()
if key not in cls._instances:
instance = super().__new__(cls)
instance._initialized = False
instance._scope_key = key
cls._instances[key] = instance
cls._instance = cls._instances[key]
return cls._instances[key]
def __init__(self, config=None):
if self._initialized:
return
if config is not None:
self.config = config
self.sessions: Dict[str, Dict[str, Any]] = {} # name -> {session, stack}
self.tool_cache: Dict[str, List[Dict[str, Any]]] = {}
self._connecting: Dict[str, asyncio.Future] = {}
if getattr(self, "_initialized", False):
return
if not hasattr(self, "config"):
self.config = config
self.sessions: dict[str, dict[str, Any]] = {} # name -> {session, stack}
self.tool_cache: dict[str, list[dict[str, Any]]] = {}
self._server_configs: dict[str, dict[str, Any]] = {}
self._connecting: dict[str, asyncio.Future] = {}
self._initialized = True
async def get_tools_for_llm(self, os_filter: Optional[str] = None) -> List[Dict[str, Any]]:
"""
Fetches tools from enabled MCP servers that match the OS filter.
"""
async def get_tools_for_llm(self, os_filter: str | None = None) -> list[dict[str, Any]]:
"""Fetches tools from enabled MCP servers that match the OS filter."""
if not MCP_AVAILABLE:
return []
@@ -52,29 +74,53 @@ class MCPClientManager:
mcp_config = self.config.get_effective_setting("ai", {}).get("mcp_servers", {})
else:
mcp_config = self.config.config.get("ai", {}).get("mcp_servers", {}) if hasattr(self.config, "config") else {}
except Exception:
except Exception: # noqa: BLE001
return []
if not isinstance(mcp_config, dict):
mcp_config = {}
# Invalidate cache and reconnect if server configuration changed or server was removed
current_names = set(mcp_config.keys())
for old_name in list(self._server_configs.keys()):
if old_name not in current_names:
self.tool_cache.pop(old_name, None)
old_data = self.sessions.pop(old_name, None)
if old_data and old_data.get("stack"):
with suppress(Exception):
await old_data["stack"].aclose()
self._server_configs.pop(old_name, None)
for name, cfg in mcp_config.items():
cached_cfg = self._server_configs.get(name)
if cached_cfg is not None and cached_cfg != cfg:
self.tool_cache.pop(name, None)
old_data = self.sessions.pop(name, None)
if old_data and old_data.get("stack"):
with suppress(Exception):
await old_data["stack"].aclose()
self._server_configs[name] = dict(cfg) if isinstance(cfg, dict) else cfg
async def _fetch(name, cfg):
if not cfg.get("enabled", True): return []
if not isinstance(cfg, dict) or not cfg.get("enabled", True):
return []
# Filter by OS if specified in config (primarily used for copilot strict matching)
auto_os = cfg.get("auto_load_on_os")
if os_filter is not None and auto_os and os_filter.lower() != auto_os.lower():
return []
try:
with suppress(Exception):
session = await self._ensure_connected(name, cfg)
if session:
if name in self.tool_cache: return self.tool_cache[name]
if name in self.tool_cache:
return self.tool_cache[name]
llm_tools = await self._fetch_tools_as_openai(name, session)
self.tool_cache[name] = llm_tools
return llm_tools
except Exception:
pass
return []
tasks = [ _fetch(name, cfg) for name, cfg in mcp_config.items() ]
tasks = [_fetch(name, cfg) for name, cfg in mcp_config.items()]
if tasks:
results = await asyncio.gather(*tasks)
@@ -83,8 +129,9 @@ class MCPClientManager:
return all_llm_tools
async def _ensure_connected(self, name: str, cfg: Dict[str, Any]) -> Optional[Any]:
if not MCP_AVAILABLE: return None
async def _ensure_connected(self, name: str, cfg: dict[str, Any]) -> Any | None:
if not MCP_AVAILABLE:
return None
if name in self.sessions and self.sessions[name].get("session"):
return self.sessions[name]["session"]
@@ -96,7 +143,7 @@ class MCPClientManager:
if name in self._connecting:
try:
return await asyncio.wait_for(asyncio.shield(self._connecting[name]), timeout=10.0)
except Exception:
except Exception: # noqa: BLE001
return None
loop = asyncio.get_running_loop()
@@ -117,33 +164,32 @@ class MCPClientManager:
self.sessions[name] = {"session": session, "stack": stack}
fut.set_result(session)
return session
except Exception:
except Exception: # noqa: BLE001
fut.set_result(None)
return None
finally:
if name in self._connecting:
del self._connecting[name]
async def _fetch_tools_as_openai(self, server_name: str, session: Any) -> List[Dict[str, Any]]:
async def _fetch_tools_as_openai(self, server_name: str, session: Any) -> list[dict[str, Any]]:
try:
result = await asyncio.wait_for(session.list_tools(), timeout=5.0)
openai_tools = []
for tool in result.tools:
# Use mcp_ prefix to ensure valid function name for LiteLLM/Gemini
prefixed_name = f"mcp_{server_name}__{tool.name}"
openai_tools.append({
"type": "function",
"function": {
"name": prefixed_name,
"description": f"[{server_name}] {tool.description}",
"parameters": tool.inputSchema
}
"parameters": tool.inputSchema,
},
})
return openai_tools
except Exception:
except Exception: # noqa: BLE001
return []
async def call_tool(self, full_tool_name: str, arguments: Dict[str, Any]) -> Any:
async def call_tool(self, full_tool_name: str, arguments: dict[str, Any]) -> Any:
"""Calls an MCP tool and returns text result."""
if not MCP_AVAILABLE:
return "Error: MCP SDK is not installed."
@@ -151,7 +197,7 @@ class MCPClientManager:
if "__" not in full_tool_name:
return f"Error: Tool {full_tool_name} is not a valid MCP tool."
clean_name = full_tool_name[4:] if full_tool_name.startswith("mcp_") else full_tool_name
clean_name = full_tool_name.removeprefix("mcp_")
server_name, tool_name = clean_name.split("__", 1)
if server_name not in self.sessions:
@@ -162,13 +208,24 @@ class MCPClientManager:
result = await asyncio.wait_for(session.call_tool(tool_name, arguments), timeout=60.0)
text_outputs = [content.text for content in result.content if hasattr(content, "text")]
return "\n".join(text_outputs) if text_outputs else str(result)
except Exception as e:
return f"Error calling tool {tool_name} on {server_name}: {str(e)}"
except Exception as e: # noqa: BLE001
return f"Error calling tool {tool_name} on {server_name}: {e!s}"
async def shutdown(self):
"""Close all SSE connections."""
for name, data in self.sessions.items():
"""Close all SSE connections and clean up resources."""
for data in list(self.sessions.values()):
stack = data.get("stack")
if stack:
with suppress(Exception):
await stack.aclose()
self.sessions = {}
self.sessions.clear()
self.tool_cache.clear()
self._server_configs.clear()
for fut in list(self._connecting.values()):
if not fut.done():
fut.cancel()
self._connecting.clear()
with self._lock:
if hasattr(self, "_scope_key") and self._scope_key in self._instances:
del self._instances[self._scope_key]
self._initialized = False
+606
View File
@@ -0,0 +1,606 @@
import contextlib
import logging
import re
import sys
from typing import Any, ClassVar
import yaml
from .auth_utils import (
MAX_FALLBACK_ENTRIES,
MAX_JSON_INPUT_BYTES,
MAX_JSON_INPUT_DEPTH,
SanitizedAuthError,
is_compose_expression,
mask_node_secrets,
sanitize_exception,
validate_auth_budget_tags,
validate_compose_syntax,
validate_fallback_entries,
validate_json_depth_and_size,
)
try:
from mcp.server.mcpserver import MCPServer
except ImportError:
try:
from mcp.server.fastmcp import FastMCP as MCPServer
except ImportError:
MCPServer = None
logger = logging.getLogger("connpy.mcp_server")
class ConnpyMCPServer:
"""Native Model Context Protocol (MCP) Server for connpy."""
SAFE_COMMANDS: ClassVar[list[str]] = [
r"^show\s+",
r"^ls\s*",
r"^cat\s+",
r"^ip\s+",
r"^pwd$",
r"^hostname$",
r"^uname",
r"^df\s*",
r"^free\s*",
r"^ps\s*",
r"^ping\s+",
r"^traceroute\s+",
r"^whois\s+",
r"^kubectl\s+(get|describe|version|logs|top|explain|cluster-info|api-resources|api-versions)\s+",
r"^systemctl\s+status\s+",
r"^journalctl\s+",
]
def __init__(self, config=None, services=None, trust: bool = False):
self.config = config
self.services = services
if self.services is None:
if self.config is not None:
from .services.provider import ServiceProvider
self.services = ServiceProvider(self.config)
elif self.config is None and hasattr(self.services, "config"):
self.config = self.services.config
self.trust = bool(trust)
# Build list of safe commands (SAFE_COMMANDS + trusted_commands from config)
aiconfig = {}
if self.config:
if hasattr(self.config, "get_effective_setting"):
aiconfig = self.config.get_effective_setting("ai", {})
elif hasattr(self.config, "config") and isinstance(self.config.config, dict):
aiconfig = self.config.config.get("ai", {})
custom_trusted = aiconfig.get("trusted_commands", []) if isinstance(aiconfig, dict) else []
if isinstance(custom_trusted, str):
custom_trusted = [c.strip() for c in custom_trusted.split(",") if c.strip()]
self.safe_commands = list(self.SAFE_COMMANDS) + (
custom_trusted if isinstance(custom_trusted, list) else []
)
# Initialize MCP Server
if MCPServer is not None:
self.server = MCPServer("connpy")
self._register_tools()
else:
self.server = None
def _register_tools(self):
"""Register the 3 tiers of MCP tools."""
if not self.server:
return
# Level 1 Tools (Read-Only)
self.server.tool()(self.list_nodes)
self.server.tool()(self.list_folders)
self.server.tool()(self.get_node_info)
self.server.tool()(self.list_profiles)
self.server.tool()(self.get_active_context)
# Level 2 Tools (Execution, Testing, Playbooks)
self.server.tool()(self.run_commands)
self.server.tool()(self.test_commands)
self.server.tool()(self.validate_playbook)
self.server.tool()(self.run_playbook)
# Level 3 Tools (CRUD - Requires trust=True)
self.server.tool()(self.add_node)
self.server.tool()(self.update_node)
self.server.tool()(self.delete_node)
# -------------------------------------------------------------------------
# Security Validation
# -------------------------------------------------------------------------
def is_safe_command(self, cmd: str) -> bool:
"""Validate if a command is considered safe under current trust settings.
- If trust is True, all commands are allowed.
- Block chaining/injection characters: ';', '&&', '||', '|', '`', '$(', '>', '<', '\\n', '\\r', '&'.
- Validates against self.safe_commands (SAFE_COMMANDS + trusted_commands).
"""
if self.trust:
return True
if not isinstance(cmd, str):
cmd = str(cmd)
# Check for shell chaining and redirection injection tokens
injection_tokens = [";", "&&", "||", "|", "`", "$(", ">", "<", "\n", "\r", "&"]
if any(tok in cmd for tok in injection_tokens):
return False
clean_cmd = cmd.strip()
return any(re.match(pattern, clean_cmd, re.IGNORECASE) for pattern in self.safe_commands)
def _mask_secrets(self, val: Any) -> Any:
"""Recursively mask passwords, passphrases, secrets, and auth tokens with '***'."""
return mask_node_secrets(val)
# -------------------------------------------------------------------------
# Level 1: Read-Only Inventory and Context Tools
# -------------------------------------------------------------------------
def list_nodes(self, filter_pattern: str = "", format_str: str = "") -> list[str]:
"""List inventory nodes, optionally filtered by regex pattern and formatted."""
if not self.services or not hasattr(self.services, "nodes"):
return []
f = filter_pattern if filter_pattern else None
fmt = format_str if format_str else None
return self.services.nodes.list_nodes(filter_str=f, format_str=fmt)
def list_folders(self, filter_pattern: str = "") -> list[str]:
"""List inventory folders, optionally filtered by regex pattern."""
if not self.services or not hasattr(self.services, "nodes"):
return []
f = filter_pattern if filter_pattern else None
return self.services.nodes.list_folders(filter_str=f)
def get_node_info(self, node_name: str) -> dict:
"""Get full configuration metadata for a specific node.
Passwords, passphrases, and secrets are ALWAYS masked with '***'.
"""
if not self.services or not hasattr(self.services, "nodes"):
return {"error": "Node service unavailable."}
try:
details = self.services.nodes.get_node_details(node_name)
if hasattr(self.services, "profiles"):
with contextlib.suppress(Exception):
details = self.services.profiles.resolve_node_data(details)
return self._mask_secrets(details)
except Exception as e: # noqa: BLE001
return {"error": f"Node '{node_name}' not found: {e}"}
def list_profiles(self, filter_pattern: str = "") -> list[str]:
"""List all available node connection profiles."""
if not self.services or not hasattr(self.services, "profiles"):
return []
f = filter_pattern if filter_pattern else None
return self.services.profiles.list_profiles(filter_str=f)
def get_active_context(self) -> dict:
"""Get the currently active context name and its matching regex patterns."""
if not self.services or not hasattr(self.services, "context"):
return {"context": "all", "patterns": [".*"]}
try:
ctx_name = self.services.context.current_context
patterns = [p.pattern for p in self.services.context.get_active_regexes()]
return {"context": ctx_name, "patterns": patterns}
except Exception as e: # noqa: BLE001
return {"context": "unknown", "error": str(e)}
# -------------------------------------------------------------------------
# Level 2: Execution, Testing, and Playbooks Tools
# -------------------------------------------------------------------------
@staticmethod
def _parse_json_list_or_lines(raw_val: list[str] | str, *, allow_empty_str: bool = False) -> list[str]:
if isinstance(raw_val, str):
if len(raw_val.encode("utf-8", errors="replace")) > MAX_JSON_INPUT_BYTES:
raise ValueError(f"Input exceeds maximum size of {MAX_JSON_INPUT_BYTES} bytes")
stripped = raw_val.strip()
if stripped.startswith(("[", "{")):
parsed = validate_json_depth_and_size(
stripped,
max_bytes=MAX_JSON_INPUT_BYTES,
max_depth=MAX_JSON_INPUT_DEPTH,
)
return parsed if isinstance(parsed, list) else [raw_val]
return ([raw_val] if raw_val else []) if allow_empty_str else [
c.strip() for c in raw_val.splitlines() if c.strip()
]
if isinstance(raw_val, list):
validate_json_depth_and_size(
raw_val,
max_bytes=MAX_JSON_INPUT_BYTES,
max_depth=MAX_JSON_INPUT_DEPTH,
)
return raw_val
return [str(raw_val)] if (raw_val or not allow_empty_str) else []
def run_commands(
self,
nodes_filter: str,
commands: list[str] | str,
parallel: bool = True,
timeout: int = 30,
) -> dict:
"""Execute commands on nodes matching the filter pattern.
Unsafe commands are blocked unless trust mode is enabled.
"""
if not self.services or not hasattr(self.services, "execution"):
return {"success": False, "error": "Execution service unavailable."}
try:
commands = self._parse_json_list_or_lines(commands, allow_empty_str=False)
except ValueError as exc:
return {"success": False, "error": str(exc)}
# Check safety
if not self.trust:
unsafe = [cmd for cmd in commands if not self.is_safe_command(cmd)]
if unsafe:
return {
"success": False,
"error": f"Permission denied: unsafe commands detected: {unsafe}. Requires trust=True.",
}
par_count = 10 if parallel is True else (parallel if isinstance(parallel, int) else 1)
try:
results = self.services.execution.run_commands(
nodes_filter=nodes_filter,
commands=commands,
parallel=par_count,
timeout=timeout,
)
fb_used = False
if isinstance(results, dict):
fb_used = any(
isinstance(v, dict) and bool(v.get("fallback_used", False))
for v in results.values()
)
if not fb_used:
fb_map = getattr(self.services.execution, "last_fallback_used", {})
if isinstance(fb_map, dict):
fb_used = any(bool(v) for v in fb_map.values())
return {"success": True, "results": results, "fallback_used": fb_used}
except Exception as e: # noqa: BLE001
san = sanitize_exception(
e,
code="AUTH_FAILED",
context_label="command execution failed",
)
return {"success": False, "error": str(san)}
def test_commands(
self,
nodes_filter: str,
commands: list[str] | str,
expected: list[str] | str = "",
parallel: bool = True,
timeout: int = 30,
) -> dict:
"""Execute commands and assert expected patterns on nodes matching the filter."""
if not self.services or not hasattr(self.services, "execution"):
return {"success": False, "error": "Execution service unavailable."}
try:
commands = self._parse_json_list_or_lines(commands, allow_empty_str=False)
expected = self._parse_json_list_or_lines(expected, allow_empty_str=True)
except ValueError as exc:
return {"success": False, "error": str(exc)}
# Check safety
if not self.trust:
unsafe = [cmd for cmd in commands if not self.is_safe_command(cmd)]
if unsafe:
return {
"success": False,
"error": f"Permission denied: unsafe commands detected: {unsafe}. Requires trust=True.",
}
par_count = 10 if parallel is True else (parallel if isinstance(parallel, int) else 1)
try:
results = self.services.execution.test_commands(
nodes_filter=nodes_filter,
commands=commands,
expected=expected,
parallel=par_count,
timeout=timeout,
)
fb_map = getattr(self.services.execution, "last_fallback_used", {})
fb_used = any(bool(v) for v in fb_map.values()) if isinstance(fb_map, dict) else False
return {"success": True, "results": results, "fallback_used": fb_used}
except Exception as e: # noqa: BLE001
san = sanitize_exception(
e,
code="AUTH_FAILED",
context_label="test execution failed",
)
return {"success": False, "error": str(san)}
def validate_playbook(self, playbook_yaml: str) -> dict:
"""Validate the syntax, schema, and tasks of a Connpy YAML playbook."""
try:
data = yaml.safe_load(playbook_yaml)
except Exception as e: # noqa: BLE001
return {"valid": False, "error": f"YAML Syntax Error: {e}"}
if not isinstance(data, dict):
return {"valid": False, "error": "Playbook must be a YAML dictionary."}
if "tasks" not in data:
return {"valid": False, "error": "Playbook missing mandatory root 'tasks' key."}
tasks = data["tasks"]
if not isinstance(tasks, list):
return {"valid": False, "error": "'tasks' must be a list of tasks."}
for idx, task in enumerate(tasks):
if not isinstance(task, dict):
return {"valid": False, "error": f"Task index {idx} must be a dictionary."}
name = task.get("name", f"Task {idx}")
mandatory = ["name", "action", "nodes", "commands", "output"]
missing = [f for f in mandatory if f not in task]
if missing:
return {
"valid": False,
"error": f"Task '{name}' (index {idx}) is missing mandatory fields: {missing}",
}
nodes = task["nodes"]
if not isinstance(nodes, (str, list)):
return {
"valid": False,
"error": f"Task '{name}' (index {idx}) 'nodes' must be a string or list of strings.",
}
if isinstance(nodes, list):
for n_idx, node_item in enumerate(nodes):
if not isinstance(node_item, str):
return {
"valid": False,
"error": f"Task '{name}' (index {idx}) 'nodes' contains non-string at index {n_idx}: {node_item}",
}
action = task["action"]
if action not in ["run", "test"]:
return {
"valid": False,
"error": f"Task '{name}' (index {idx}) invalid action '{action}'. Choices: 'run', 'test'.",
}
if action == "test" and "expected" not in task:
return {
"valid": False,
"error": f"Task '{name}' (index {idx}) action 'test' requires 'expected' key.",
}
output = task["output"]
if output not in [None, "stdout", "null"] and not str(output).startswith("/"):
return {
"valid": False,
"error": f"Task '{name}' (index {idx}) output '{output}' must be 'stdout', 'null' or an absolute path.",
}
return {"valid": True, "message": "Playbook schema and syntax is valid."}
def run_playbook(self, playbook_yaml: str) -> dict:
"""Validate and execute a playbook, verifying commands beforehand if trust=False."""
val = self.validate_playbook(playbook_yaml)
if not val.get("valid"):
return {"success": False, "error": f"Invalid playbook: {val.get('error')}"}
data = yaml.safe_load(playbook_yaml)
tasks = data.get("tasks", [])
# Pre-verify commands if not trusted
if not self.trust:
for idx, task in enumerate(tasks):
name = task.get("name", f"Task {idx}")
commands = task.get("commands", [])
if isinstance(commands, str):
commands = [commands]
unsafe = [cmd for cmd in commands if not self.is_safe_command(cmd)]
if unsafe:
return {
"success": False,
"error": f"Permission denied: unsafe commands detected in task '{name}': {unsafe}. Requires trust=True.",
}
# Execute tasks
results = {}
for idx, task in enumerate(tasks):
name = task.get("name", f"task_{idx}")
action = task.get("action")
nodes = task.get("nodes")
commands = task.get("commands")
if isinstance(commands, str):
commands = [commands]
try:
if action == "run":
res = self.services.execution.run_commands(nodes_filter=nodes, commands=commands)
elif action == "test":
expected = task.get("expected", [])
if isinstance(expected, str):
expected = [expected]
res = self.services.execution.test_commands(
nodes_filter=nodes, commands=commands, expected=expected
)
else:
res = f"Unsupported action: {action}"
results[name] = res
except Exception as e: # noqa: BLE001
return {
"success": False,
"error": f"Execution failed at task '{name}': {e}",
"partial_results": results,
}
return {"success": True, "results": results}
# -------------------------------------------------------------------------
# Level 3: Inventory Mutation Tools (CRUD - Requires trust=True)
# -------------------------------------------------------------------------
@staticmethod
def _validate_and_normalize_node_payload(node_data: dict[str, Any]) -> dict[str, Any]:
validate_json_depth_and_size(
node_data,
max_bytes=MAX_JSON_INPUT_BYTES,
max_depth=MAX_JSON_INPUT_DEPTH,
)
normalized = node_data.copy()
if "tags" in normalized and normalized["tags"] not in ("", None):
tags_val = normalized["tags"]
if isinstance(tags_val, str) and tags_val.strip().startswith("{"):
tags_val = validate_json_depth_and_size(
tags_val,
max_bytes=MAX_JSON_INPUT_BYTES,
max_depth=MAX_JSON_INPUT_DEPTH,
)
normalized["tags"] = tags_val
validate_auth_budget_tags(tags_val)
if "fallback" in normalized and normalized["fallback"] not in (None, "", []):
fb_val = normalized["fallback"]
if isinstance(fb_val, str) and fb_val.strip().startswith(("[", "{")):
fb_val = validate_json_depth_and_size(
fb_val,
max_bytes=MAX_JSON_INPUT_BYTES,
max_depth=MAX_JSON_INPUT_DEPTH,
)
normalized["fallback"] = validate_fallback_entries(
fb_val,
max_entries=MAX_FALLBACK_ENTRIES,
)
pw = normalized.get("password")
if is_compose_expression(pw):
validate_compose_syntax(pw)
elif isinstance(pw, (list, tuple)):
for item in pw:
if is_compose_expression(item):
validate_compose_syntax(item)
return normalized
def add_node(
self, unique_id: str, data: dict | None = None, is_folder: bool = False, **kwargs
) -> dict:
"""Add a new node or folder to the inventory (requires trust=True)."""
if not self.trust:
return {
"success": False,
"error": "Permission denied: CRUD operations require trust mode (--trust).",
}
if not self.services or not hasattr(self.services, "nodes"):
return {"success": False, "error": "Node service unavailable."}
node_data = (data or {}).copy()
node_data.update(kwargs)
try:
node_data = self._validate_and_normalize_node_payload(node_data)
self.services.nodes.add_node(unique_id, node_data, is_folder=is_folder)
return {
"success": True,
"message": f"{'Folder' if is_folder else 'Node'} '{unique_id}' created successfully.",
}
except (ValueError, TypeError, SanitizedAuthError) as exc:
return {"success": False, "error": str(exc)}
except Exception as e: # noqa: BLE001
return {"success": False, "error": str(e)}
def update_node(self, unique_id: str, updates: dict | None = None, **kwargs) -> dict:
"""Update an existing node configuration (requires trust=True)."""
if not self.trust:
return {
"success": False,
"error": "Permission denied: CRUD operations require trust mode (--trust).",
}
if not self.services or not hasattr(self.services, "nodes"):
return {"success": False, "error": "Node service unavailable."}
node_updates = (updates or {}).copy()
node_updates.update(kwargs)
try:
node_updates = self._validate_and_normalize_node_payload(node_updates)
self.services.nodes.update_node(unique_id, node_updates)
return {"success": True, "message": f"Node '{unique_id}' updated successfully."}
except (ValueError, TypeError, SanitizedAuthError) as exc:
return {"success": False, "error": str(exc)}
except Exception as e: # noqa: BLE001
return {"success": False, "error": str(e)}
def delete_node(self, unique_id: str, is_folder: bool = False) -> dict:
"""Delete a node or folder from the inventory (requires trust=True)."""
if not self.trust:
return {
"success": False,
"error": "Permission denied: CRUD operations require trust mode (--trust).",
}
if not self.services or not hasattr(self.services, "nodes"):
return {"success": False, "error": "Node service unavailable."}
try:
self.services.nodes.delete_node(unique_id, is_folder=is_folder)
return {
"success": True,
"message": f"{'Folder' if is_folder else 'Node'} '{unique_id}' deleted successfully.",
}
except Exception as e: # noqa: BLE001
return {"success": False, "error": str(e)}
# -------------------------------------------------------------------------
# Server Execution and Async Helper
# -------------------------------------------------------------------------
async def call_tool(self, name: str, arguments: dict | None = None) -> Any:
"""Call an MCP tool asynchronously via the underlying MCP server."""
if not self.server:
raise RuntimeError("MCP SDK is not initialized.")
return await self.server.call_tool(name, arguments or {})
def run(self, transport: str = "stdio", host: str = "127.0.0.1", port: int = 8000, **kwargs):
"""Run the MCP server using either stdio or sse transport."""
if not self.server:
raise RuntimeError("MCP server cannot start: 'mcp' package is not installed.")
if transport == "stdio":
# Redirect logs and printer outputs to sys.stderr for JSON-RPC
with contextlib.suppress(Exception):
from . import printer
printer.set_thread_stream(sys.stderr)
for handler in logging.root.handlers:
if hasattr(handler, "stream") and handler.stream == sys.stdout:
handler.stream = sys.stderr
logging.basicConfig(stream=sys.stderr, level=logging.INFO)
return self.server.run(transport="stdio", **kwargs)
if transport == "sse":
if hasattr(self.server, "settings"):
if hasattr(self.server.settings, "host"):
self.server.settings.host = host
if hasattr(self.server.settings, "port"):
self.server.settings.port = port
try:
return self.server.run(transport="sse", host=host, port=port, **kwargs)
except TypeError:
return self.server.run(transport="sse", **kwargs)
msg = f"Unknown transport: {transport}. Supported transports: 'stdio', 'sse'"
raise ValueError(msg)
+72 -14
View File
@@ -1,21 +1,45 @@
import re
from typing import List, Dict, Any
from typing import Any
from ..auth_utils import (
MAX_CONTEXT_PATTERN_LEN,
MAX_CONTEXT_REGEXES,
compile_active_context_patterns,
is_default_all_context_cfg,
match_active_context_patterns,
)
from .base import BaseService
from ..hooks import MethodHook
from .. import printer
from .exceptions import InvalidConfigurationError
class ContextService(BaseService):
"""Business logic for managing and applying regex-based contexts locally."""
@staticmethod
def _validate_regexes(regexes: Any) -> None:
if not isinstance(regexes, list) or not (1 <= len(regexes) <= MAX_CONTEXT_REGEXES):
raise ValueError(
f"Context regexes must be a list of 1 to {MAX_CONTEXT_REGEXES} patterns"
)
for pat in regexes:
if not isinstance(pat, str) or not (1 <= len(pat) <= MAX_CONTEXT_PATTERN_LEN):
raise ValueError(
f"Each context regex must be a non-empty string up to {MAX_CONTEXT_PATTERN_LEN} characters"
)
try:
re.compile(pat)
except re.error as exc:
raise ValueError(f"Invalid context regex {pat!r}: {exc}") from None
@property
def contexts(self) -> Dict[str, List[str]]:
def contexts(self) -> dict[str, list[str]]:
return self.config.config.get("contexts", {"all": [".*"]})
@property
def current_context(self) -> str:
return self.config.config.get("current_context", "all")
def list_contexts(self) -> List[Dict[str, Any]]:
def list_contexts(self) -> list[dict[str, Any]]:
result = []
for name in self.contexts.keys():
result.append({
@@ -25,27 +49,29 @@ class ContextService(BaseService):
})
return result
def add_context(self, name: str, regexes: List[str]):
def add_context(self, name: str, regexes: list[str]):
if not name.isalnum():
raise ValueError("Context name must be alphanumeric")
self._validate_regexes(regexes)
ctxs = self.contexts
if name in ctxs:
raise ValueError(f"Context '{name}' already exists")
ctxs[name] = regexes
ctxs[name] = list(regexes)
self.config.config["contexts"] = ctxs
self.config._saveconfig(self.config.file)
def update_context(self, name: str, regexes: List[str]):
def update_context(self, name: str, regexes: list[str]):
if name == "all":
raise ValueError("Cannot modify default context 'all'")
self._validate_regexes(regexes)
ctxs = self.contexts
if name not in ctxs:
raise ValueError(f"Context '{name}' does not exist")
ctxs[name] = regexes
ctxs[name] = list(regexes)
self.config.config["contexts"] = ctxs
self.config._saveconfig(self.config.file)
@@ -67,15 +93,47 @@ class ContextService(BaseService):
if name not in self.contexts:
raise ValueError(f"Context '{name}' does not exist")
prev_context = self.current_context
self.config.config["current_context"] = name
self.config._saveconfig(self.config.file)
can_refresh = (
self.config._is_context_filtering_enabled_unlocked()
if (
not hasattr(self.config, "assert_called")
and callable(getattr(self.config, "_is_context_filtering_enabled_unlocked", None))
)
else getattr(self.config, "_enforce_context", True)
)
if (
name == "all"
and prev_context != "all"
and can_refresh
and hasattr(self.config, "_generate_nodes_cache")
):
self.config._generate_nodes_cache()
def get_active_regexes(self) -> List[re.Pattern]:
patterns = self.contexts.get(self.current_context, [".*"])
return [re.compile(p) for p in patterns]
def get_active_regexes(self) -> list[re.Pattern]:
return compile_active_context_patterns(getattr(self.config, "config", None))
def _match_any(self, node_name: str, patterns: List[re.Pattern]) -> bool:
return any(p.match(node_name) for p in patterns)
def _match_any(self, node_name: str, patterns: list[re.Pattern]) -> bool:
return match_active_context_patterns(node_name, patterns)
def is_default_all_context(self) -> bool:
return is_default_all_context_cfg(getattr(self.config, "config", None))
def matches_active_context(self, name: str) -> bool:
if self.is_default_all_context():
return isinstance(name, str) and bool(name)
if not isinstance(name, str) or not name:
return False
return match_active_context_patterns(name, self.get_active_regexes())
def validate_in_active_context(self, unique_id: str, is_folder: bool = False) -> None:
if not self.is_default_all_context() and not self.matches_active_context(unique_id):
label = "Folder" if is_folder else "Node"
raise InvalidConfigurationError(
f"{label} '{unique_id}' does not match active context '{self.current_context}'."
)
# Hook handlers for filtering
def filter_node_list(self, *args, **kwargs):
+308 -55
View File
@@ -1,29 +1,215 @@
from typing import List, Dict, Any, Callable, Optional
import inspect
import os
from .base import BaseService
import re
import sys
from collections.abc import Callable
from contextlib import suppress
from typing import Any
from connpy.auth_utils import (
MAX_REMOTE_FILTER_NODES,
AuthAttemptTracker,
raise_sanitized_auth_error,
)
from connpy.core import nodes as Nodes
from .base import BaseService
from .exceptions import ConnpyError
class ExecutionService(BaseService):
"""Business logic for executing commands on nodes and running automation scripts."""
def __init__(
self,
config=None,
remote_stub=None,
attempt_tracker=None,
enforce_context: bool = True,
):
super().__init__(config)
self.remote_stub = remote_stub
self.attempt_tracker = (
attempt_tracker if attempt_tracker is not None else AuthAttemptTracker()
)
self.enforce_context = bool(enforce_context)
self.last_fallback_used: dict[str, bool] = {}
def _get_context_service(self):
if (
not getattr(self, "enforce_context", True)
or not getattr(self.config, "_enforce_context", True)
or (len(sys.argv) > 1 and sys.argv[1] == "api")
or not isinstance(getattr(self.config, "config", None), dict)
):
return None
from .context_service import ContextService
return ContextService(self.config)
def _is_remote_backend(self) -> bool:
"""Return True when execution is configured to run against a remote gRPC backend."""
if getattr(self, "remote_stub", None) is not None:
return True
cfg = getattr(self.config, "config", None)
return isinstance(cfg, dict) and cfg.get("service_mode") == "remote"
def _ensure_backend_supports_stage_timeouts(
self,
connect_timeout: int | None = None,
auth_timeout: int | None = None,
) -> None:
"""Raise ConnpyError if per-stage timeout overrides are used on gRPC remote backend."""
if self._is_remote_backend() and (connect_timeout is not None or auth_timeout is not None):
raise ConnpyError(
"connect_timeout and auth_timeout overrides are not supported in remote gRPC mode"
)
def _get_remote_channel(self):
cfg = getattr(self.config, "config", None)
remote_host = cfg.get("remote_host") if isinstance(cfg, dict) else None
if not remote_host:
raise ConnpyError("Remote host must be specified in remote mode")
import grpc
from connpy.grpc_layer.stubs import AuthClientInterceptor
def get_token():
env_token = os.environ.get("CONNPY_TOKEN")
if env_token:
return env_token
defaultdir = getattr(self.config, "defaultdir", None)
if defaultdir:
token_path = os.path.join(defaultdir, ".token")
if os.path.exists(token_path):
with suppress(OSError), open(token_path) as f:
return f.read().strip()
return None
channel = grpc.insecure_channel(remote_host)
channel = grpc.intercept_channel(channel, AuthClientInterceptor(get_token))
return channel, remote_host
def _get_remote_stub(self):
"""Return the configured or lazily constructed ExecutionStub for remote gRPC mode."""
if getattr(self, "remote_stub", None) is not None:
return self.remote_stub
from connpy.grpc_layer.stubs import ExecutionStub
channel, remote_host = self._get_remote_channel()
return ExecutionStub(channel, remote_host=remote_host)
def _filter_remote_nodes_filter(self, nodes_filter):
ctx_svc = self._get_context_service()
if ctx_svc is None or ctx_svc.is_default_all_context():
return nodes_filter
if isinstance(nodes_filter, (list, tuple)):
permitted = ctx_svc.filter_node_list(result=list(nodes_filter))
else:
if self.remote_stub is not None and callable(
getattr(self.remote_stub, "list_nodes", None)
):
candidates = self.remote_stub.list_nodes(filter_str=nodes_filter)
elif self.remote_stub is not None and callable(
getattr(getattr(self.remote_stub, "node_stub", None), "list_nodes", None)
):
candidates = self.remote_stub.node_stub.list_nodes(filter_str=nodes_filter)
elif self.remote_stub is None:
from connpy.grpc_layer.stubs import NodeStub
channel, remote_host = self._get_remote_channel()
node_stub = NodeStub(channel, remote_host=remote_host, config=self.config)
candidates = node_stub.list_nodes(filter_str=nodes_filter)
else:
candidates = (
self.config._getallnodes(nodes_filter)
if hasattr(self.config, "_getallnodes")
else []
)
if nodes_filter:
cfg_dict = getattr(self.config, "config", None)
case = cfg_dict.get("case", False) if isinstance(cfg_dict, dict) else False
flags = 0 if case else re.IGNORECASE
candidates = [c for c in candidates if re.search(nodes_filter, c, flags)]
permitted = ctx_svc.filter_node_list(result=candidates)
if not permitted:
raise ConnpyError(f"No nodes found matching filter: {nodes_filter}")
return [f"^{re.escape(n)}$" for n in permitted]
def _build_remote_filter_batches(self, nodes_filter):
filtered = self._filter_remote_nodes_filter(nodes_filter)
if isinstance(filtered, list):
return [
filtered[i : i + MAX_REMOTE_FILTER_NODES]
for i in range(0, len(filtered), MAX_REMOTE_FILTER_NODES)
]
return [filtered]
@staticmethod
def _aggregate_remote_results(batch_results):
if len(batch_results) == 1:
return batch_results[0]
if all(isinstance(r, dict) for r in batch_results):
merged: dict[str, Any] = {}
for r in batch_results:
merged.update(r)
return merged
if all(isinstance(r, list) for r in batch_results):
merged_list: list[Any] = []
for r in batch_results:
merged_list.extend(r)
return merged_list
return batch_results[-1]
def run_commands(
self,
nodes_filter: str,
commands: List[str],
variables: Optional[Dict[str, Any]] = None,
commands: list[str],
variables: dict[str, Any] | None = None,
parallel: int = 10,
timeout: int = 20,
folder: Optional[str] = None,
prompt: Optional[str] = None,
on_node_complete: Optional[Callable] = None,
logger: Optional[Callable] = None,
name: Optional[str] = None
) -> Dict[str, str]:
timeout: int | None = None,
folder: str | None = None,
prompt: str | None = None,
on_node_complete: Callable | None = None,
logger: Callable | None = None,
name: str | None = None,
connect_timeout: int | None = None,
auth_timeout: int | None = None,
) -> dict[str, Any]:
"""Execute commands on a set of nodes."""
try:
self._ensure_backend_supports_stage_timeouts(
connect_timeout=connect_timeout,
auth_timeout=auth_timeout,
)
if self._is_remote_backend():
batches = self._build_remote_filter_batches(nodes_filter)
stub = self._get_remote_stub()
batch_results = [
stub.run_commands(
nodes_filter=batch,
commands=commands,
variables=variables,
parallel=parallel,
timeout=timeout if timeout is not None else 20,
folder=folder,
prompt=prompt,
on_node_complete=on_node_complete,
logger=logger,
name=name or "",
)
for batch in batches
]
return self._aggregate_remote_results(batch_results)
matched_names = self.config._getallnodes(nodes_filter)
ctx_svc = self._get_context_service()
if ctx_svc is not None and not ctx_svc.is_default_all_context():
matched_names = ctx_svc.filter_node_list(result=matched_names)
if not matched_names:
raise ConnpyError(f"No nodes found matching filter: {nodes_filter}")
@@ -31,47 +217,96 @@ class ExecutionService(BaseService):
executor = Nodes(node_data, config=self.config)
self.last_executor = executor
results = executor.run(
commands=commands,
vars=variables,
parallel=parallel,
timeout=timeout,
folder=folder,
prompt=prompt,
on_complete=on_node_complete,
logger=logger
)
run_kwargs: dict[str, Any] = {
"commands": commands,
"vars": variables,
"parallel": parallel,
"timeout": timeout,
"folder": folder,
"prompt": prompt,
"on_complete": on_node_complete,
"logger": logger,
"connect_timeout": connect_timeout,
"auth_timeout": auth_timeout,
"_default_cmd_timeout": 20,
}
run_params = inspect.signature(executor.run).parameters
if "unattended" in run_params:
run_kwargs["unattended"] = True
if "attempt_tracker" in run_params:
run_kwargs["attempt_tracker"] = self.attempt_tracker
# Combine output and status for the caller
results = executor.run(**run_kwargs)
fb_map = getattr(executor, "fallback_used", {})
fb_dict = fb_map if isinstance(fb_map, dict) else {}
# Combine output, status, and fallback metadata for the caller
full_results = {}
self.last_fallback_used = {}
for unique in results:
fb_used = bool(fb_dict.get(unique, False))
full_results[unique] = {
"output": results[unique],
"status": executor.status.get(unique, 1)
"status": executor.status.get(unique, 1),
"fallback_used": fb_used,
}
self.last_fallback_used[unique] = fb_used
return full_results
except Exception as e:
raise ConnpyError(f"Execution failed: {e}")
except ConnpyError:
raise
except Exception as e: # noqa: BLE001
raise_sanitized_auth_error(e, code="AUTH_FAILED", context_label="execution failed")
def test_commands(
self,
nodes_filter: str,
commands: List[str],
expected: List[str],
variables: Optional[Dict[str, Any]] = None,
commands: list[str],
expected: list[str],
variables: dict[str, Any] | None = None,
parallel: int = 10,
timeout: int = 20,
folder: Optional[str] = None,
prompt: Optional[str] = None,
on_node_complete: Optional[Callable] = None,
logger: Optional[Callable] = None,
name: Optional[str] = None
) -> Dict[str, Dict[str, bool]]:
timeout: int | None = None,
folder: str | None = None,
prompt: str | None = None,
on_node_complete: Callable | None = None,
logger: Callable | None = None,
name: str | None = None,
connect_timeout: int | None = None,
auth_timeout: int | None = None,
) -> dict[str, dict[str, bool]]:
"""Run commands and verify expected output on a set of nodes."""
try:
self._ensure_backend_supports_stage_timeouts(
connect_timeout=connect_timeout,
auth_timeout=auth_timeout,
)
if self._is_remote_backend():
batches = self._build_remote_filter_batches(nodes_filter)
stub = self._get_remote_stub()
batch_results = [
stub.test_commands(
nodes_filter=batch,
commands=commands,
expected=expected,
variables=variables,
parallel=parallel,
timeout=timeout if timeout is not None else 20,
folder=folder or "",
prompt=prompt,
on_node_complete=on_node_complete,
logger=logger,
name=name or "",
)
for batch in batches
]
return self._aggregate_remote_results(batch_results)
matched_names = self.config._getallnodes(nodes_filter)
ctx_svc = self._get_context_service()
if ctx_svc is not None and not ctx_svc.is_default_all_context():
matched_names = ctx_svc.filter_node_list(result=matched_names)
if not matched_names:
raise ConnpyError(f"No nodes found matching filter: {nodes_filter}")
@@ -79,31 +314,49 @@ class ExecutionService(BaseService):
executor = Nodes(node_data, config=self.config)
self.last_executor = executor
results = executor.test(
commands=commands,
expected=expected,
vars=variables,
parallel=parallel,
timeout=timeout,
folder=folder,
prompt=prompt,
on_complete=on_node_complete,
logger=logger
)
return results
except Exception as e:
raise ConnpyError(f"Testing failed: {e}")
test_kwargs: dict[str, Any] = {
"commands": commands,
"expected": expected,
"vars": variables,
"parallel": parallel,
"timeout": timeout,
"folder": folder,
"prompt": prompt,
"on_complete": on_node_complete,
"logger": logger,
"connect_timeout": connect_timeout,
"auth_timeout": auth_timeout,
"_default_cmd_timeout": 20,
}
test_params = inspect.signature(executor.test).parameters
if "unattended" in test_params:
test_kwargs["unattended"] = True
if "attempt_tracker" in test_params:
test_kwargs["attempt_tracker"] = self.attempt_tracker
def run_cli_script(self, nodes_filter: str, script_path: str, parallel: int = 10) -> Dict[str, str]:
res = executor.test(**test_kwargs)
fb_map = getattr(executor, "fallback_used", {})
fb_dict = fb_map if isinstance(fb_map, dict) else {}
if isinstance(res, dict):
self.last_fallback_used = {u: bool(fb_dict.get(u, False)) for u in res}
else:
self.last_fallback_used = {}
return res
except ConnpyError:
raise
except Exception as e: # noqa: BLE001
raise_sanitized_auth_error(e, code="AUTH_FAILED", context_label="testing failed")
def run_cli_script(self, nodes_filter: str, script_path: str, parallel: int = 10) -> dict[str, str]:
"""Run a plain-text script containing one command per line."""
if not os.path.exists(script_path):
raise ConnpyError(f"Script file not found: {script_path}")
try:
with open(script_path, "r") as f:
with open(script_path) as f:
commands = [line.strip() for line in f if line.strip()]
except Exception as e:
raise ConnpyError(f"Failed to read script {script_path}: {e}")
except OSError:
raise ConnpyError("Failed to read script file") from None
return self.run_commands(nodes_filter, commands, parallel=parallel)
+147 -27
View File
@@ -1,14 +1,56 @@
from .base import BaseService
import yaml
import os
import sys
from copy import deepcopy
from .exceptions import InvalidConfigurationError, NodeNotFoundError, ReservedNameError
import yaml
from ..configfile import NoAliasDumper
from .base import BaseService
from .exceptions import InvalidConfigurationError, NodeNotFoundError
from .node_service import NodeService, _supports_transactional_update
class ImportExportService(BaseService):
"""Business logic for YAML/JSON inventory import and export."""
def __init__(self, config=None, enforce_context: bool = True):
super().__init__(config)
self.enforce_context = bool(enforce_context)
def _get_context_service(self, cfg=None):
target_cfg = cfg or self.config
if (
not getattr(self, "enforce_context", True)
or not getattr(target_cfg, "_enforce_context", True)
or (len(sys.argv) > 1 and sys.argv[1] == "api")
or not isinstance(getattr(target_cfg, "config", None), dict)
):
return None
from .context_service import ContextService
return ContextService(target_cfg)
def _get_raw_folders(self, cfg=None) -> list[str]:
target_cfg = cfg or self.config
if target_cfg is None:
return []
if (
not hasattr(target_cfg, "assert_called")
and callable(getattr(target_cfg, "_getallfolders_unlocked", None))
and hasattr(target_cfg, "_mutation_lock")
):
with target_cfg._mutation_lock:
return target_cfg._getallfolders_unlocked()
return target_cfg._getallfolders()
def _folder_exists(self, folder_id: str, raw_folders: list[str], cfg=None) -> bool:
target_cfg = cfg or self.config
cfg_dict = getattr(target_cfg, "config", None)
case = cfg_dict.get("case", False) if isinstance(cfg_dict, dict) else False
if case:
return folder_id in raw_folders
return folder_id.lower() in {f.lower() for f in raw_folders}
def export_to_file(self, file_path, folders=None):
"""Export nodes/folders to a YAML file."""
if os.path.exists(file_path):
@@ -23,16 +65,42 @@ class ImportExportService(BaseService):
def export_to_dict(self, folders=None):
"""Export nodes/folders to a dictionary."""
if not folders:
return deepcopy(self.config.connections)
else:
# Validate folders exist
for f in folders:
if f != "@" and f not in self.config._getallfolders():
raise NodeNotFoundError(f"Folder '{f}' not found.")
ctx_svc = self._get_context_service()
ctx_active = ctx_svc is not None and not ctx_svc.is_default_all_context()
if not folders and not ctx_active:
return deepcopy(self.config.connections)
raw_folders = self._get_raw_folders()
if folders:
for f in folders:
if f != "@" and (
not self._folder_exists(f, raw_folders)
or (ctx_active and not ctx_svc.matches_active_context(f))
):
raise NodeNotFoundError(f"Folder '{f}' not found.")
flat = self.config._getallnodesfull(folders, extract=False)
allowed_folders = []
else:
allowed_folders = ctx_svc.filter_node_list(result=raw_folders)
flat = self.config._getallnodesfull(extract=False)
if ctx_active:
flat = ctx_svc.filter_node_dict(result=flat)
nested = {}
for folder_id in allowed_folders:
uniques = self.config._explode_unique(folder_id)
if not uniques or "folder" not in uniques:
continue
f_name = uniques["folder"]
if f_name not in nested:
nested[f_name] = {"type": "folder"}
if "subfolder" in uniques:
s_name = uniques["subfolder"]
if s_name not in nested[f_name]:
nested[f_name][s_name] = {"type": "subfolder"}
for k, v in flat.items():
uniques = self.config._explode_unique(k)
if not uniques:
@@ -71,28 +139,58 @@ class ImportExportService(BaseService):
try:
with open(file_path, "r") as f:
data = yaml.load(f, Loader=yaml.FullLoader)
data = yaml.safe_load(f)
self.import_from_dict(data)
except Exception as e:
raise InvalidConfigurationError(f"Failed to read/parse import file: {e}")
def import_from_dict(self, data):
"""Import nodes/folders from a dictionary."""
"""Import nodes/folders from a dictionary atomically with context pre-validation."""
if not isinstance(data, dict):
raise InvalidConfigurationError("Invalid import data format: expected a dictionary of nodes.")
raise InvalidConfigurationError(
"Invalid import data format: expected a dictionary of nodes."
)
def _traverse_import(node_data, current_folder='', current_subfolder=''):
ctx_svc = self._get_context_service()
ctx_active = ctx_svc is not None and not ctx_svc.is_default_all_context()
raw_folders = self._get_raw_folders() if ctx_active else []
planned_ops = []
def _plan_import(node_data, current_folder="", current_subfolder=""):
for k, v in node_data.items():
if k == "type":
if k == "type" or not isinstance(v, dict):
continue
if isinstance(v, dict):
node_type = v.get("type", "connection")
if node_type == "folder":
self.config._folder_add(folder=k)
_traverse_import(v, current_folder=k, current_subfolder='')
folder_id = f"@{k}"
has_children = any(
isinstance(cv, dict) for ck, cv in v.items() if ck != "type"
)
if ctx_active and (
not has_children
or (
not ctx_svc.matches_active_context(folder_id)
and not self._folder_exists(folder_id, raw_folders)
)
):
ctx_svc.validate_in_active_context(folder_id, is_folder=True)
planned_ops.append(("folder", k, ""))
_plan_import(v, current_folder=k, current_subfolder="")
elif node_type == "subfolder":
self.config._folder_add(folder=current_folder, subfolder=k)
_traverse_import(v, current_folder=current_folder, current_subfolder=k)
folder_id = f"@{k}@{current_folder}"
has_children = any(
isinstance(cv, dict) for ck, cv in v.items() if ck != "type"
)
if ctx_active and (
not has_children
or (
not ctx_svc.matches_active_context(folder_id)
and not self._folder_exists(folder_id, raw_folders)
)
):
ctx_svc.validate_in_active_context(folder_id, is_folder=True)
planned_ops.append(("folder", current_folder, k))
_plan_import(v, current_folder=current_folder, current_subfolder=k)
elif node_type == "connection":
unique_id = k
if current_subfolder:
@@ -100,16 +198,38 @@ class ImportExportService(BaseService):
elif current_folder:
unique_id = f"{k}@{current_folder}"
self._validate_node_name(unique_id)
NodeService._validate_node_payload(v)
if ctx_svc is not None:
ctx_svc.validate_in_active_context(unique_id, is_folder=False)
kwargs = deepcopy(v)
kwargs['id'] = k
kwargs['folder'] = current_folder
kwargs['subfolder'] = current_subfolder
kwargs["id"] = k
kwargs["folder"] = current_folder
kwargs["subfolder"] = current_subfolder
planned_ops.append(("connection", kwargs))
self.config._connections_add(**kwargs)
_plan_import(data)
if _supports_transactional_update(self.config):
def _mut(cfg):
for op in planned_ops:
if op[0] == "folder":
cfg._folder_add_unlocked(folder=op[1], subfolder=op[2])
else:
# Invalid format skip
pass
cfg._connections_add_unlocked(**op[1])
_traverse_import(data)
self.config.transactional_update(_mut)
return
snap_connections = deepcopy(self.config.connections)
try:
for op in planned_ops:
if op[0] == "folder":
self.config._folder_add(folder=op[1], subfolder=op[2])
else:
self.config._connections_add(**op[1])
self.config._saveconfig(self.config.file)
except BaseException:
self.config.connections.clear()
self.config.connections.update(snap_connections)
raise
+384 -33
View File
@@ -1,13 +1,169 @@
import inspect
import re
import sys
from copy import deepcopy
from typing import ClassVar
from .base import BaseService
from .exceptions import (
NodeNotFoundError, NodeAlreadyExistsError,
InvalidConfigurationError, ReservedNameError
InvalidConfigurationError,
NodeAlreadyExistsError,
NodeNotFoundError,
ReservedNameError,
)
def _supports_transactional_update(cfg):
return (
cfg is not None
and not hasattr(cfg, "assert_called")
and callable(getattr(cfg, "transactional_update", None))
)
class NodeService(BaseService):
def __init__(self, config=None):
_CONNECTION_FIELDS: ClassVar[frozenset[str]] = frozenset({
"id",
"host",
"folder",
"subfolder",
"options",
"logs",
"password",
"port",
"protocol",
"user",
"tags",
"jumphost",
"fallback",
"type",
})
def __init__(self, config=None, enforce_context: bool = True):
super().__init__(config)
self.enforce_context = bool(enforce_context)
def _get_context_service(self, cfg=None):
target_cfg = cfg or self.config
if (
not getattr(self, "enforce_context", True)
or not getattr(target_cfg, "_enforce_context", True)
or (len(sys.argv) > 1 and sys.argv[1] == "api")
or not isinstance(getattr(target_cfg, "config", None), dict)
):
return None
from .context_service import ContextService
return ContextService(target_cfg)
def _get_raw_folders(self, cfg=None) -> list[str]:
target_cfg = cfg or self.config
if target_cfg is None:
return []
if (
not hasattr(target_cfg, "assert_called")
and callable(getattr(target_cfg, "_getallfolders_unlocked", None))
and hasattr(target_cfg, "_mutation_lock")
):
with target_cfg._mutation_lock:
return target_cfg._getallfolders_unlocked()
return target_cfg._getallfolders()
def _get_raw_nodes(self, cfg=None) -> list[str]:
target_cfg = cfg or self.config
if target_cfg is None:
return []
if (
not hasattr(target_cfg, "assert_called")
and callable(getattr(target_cfg, "_getallnodes_unlocked", None))
and hasattr(target_cfg, "_mutation_lock")
):
with target_cfg._mutation_lock:
return target_cfg._getallnodes_unlocked()
return target_cfg._getallnodes()
def _derive_parent_folder(self, cfg, unique_id: str, is_folder: bool) -> str | None:
explode_fn = (
cfg._explode_unique_unlocked
if (
not hasattr(cfg, "assert_called")
and callable(getattr(cfg, "_explode_unique_unlocked", None))
)
else cfg._explode_unique
)
uniques = explode_fn(unique_id)
if is_folder:
if (
not isinstance(unique_id, str)
or not unique_id.startswith("@")
or not isinstance(uniques, dict)
or "folder" not in uniques
):
raise InvalidConfigurationError(f"Invalid folder name '{unique_id}'.")
if "subfolder" in uniques:
return f"@{uniques['folder']}"
return None
if (
not isinstance(unique_id, str)
or unique_id.startswith("@")
or not isinstance(uniques, dict)
or not uniques.get("id")
):
raise InvalidConfigurationError(f"Invalid node name '{unique_id}'.")
if "subfolder" in uniques and "folder" in uniques:
return f"@{uniques['subfolder']}@{uniques['folder']}"
if "folder" in uniques:
return f"@{uniques['folder']}"
return None
def _folder_exists(self, parent_folder: str, raw_folders: list[str], cfg=None) -> bool:
target_cfg = cfg or self.config
cfg_dict = getattr(target_cfg, "config", None)
case = cfg_dict.get("case", False) if isinstance(cfg_dict, dict) else False
if case:
return parent_folder in raw_folders
return parent_folder.lower() in {f.lower() for f in raw_folders}
def _ensure_in_active_context(
self, unique_id: str, *, is_folder: bool = False, not_found_msg: str | None = None
) -> None:
ctx_svc = self._get_context_service()
if (
ctx_svc is not None
and not ctx_svc.is_default_all_context()
and not ctx_svc.matches_active_context(unique_id)
):
if not_found_msg is not None:
raise NodeNotFoundError(not_found_msg)
label = "Folder" if is_folder else "Node"
raise NodeNotFoundError(f"{label} '{unique_id}' not found.")
@staticmethod
def _validate_node_payload(data):
if not isinstance(data, dict):
return
from ..auth_utils import (
SanitizedAuthError,
is_compose_expression,
validate_auth_budget_tags,
validate_compose_syntax,
validate_fallback_entries,
)
try:
if "tags" in data and data["tags"] not in ("", None):
validate_auth_budget_tags(data["tags"])
if "fallback" in data and data["fallback"] not in (None, "", []):
validate_fallback_entries(data["fallback"])
pw = data.get("password")
if is_compose_expression(pw):
validate_compose_syntax(pw)
elif isinstance(pw, (list, tuple)):
for item in pw:
if is_compose_expression(item):
validate_compose_syntax(item)
except (ValueError, TypeError, SanitizedAuthError) as exc:
raise InvalidConfigurationError(str(exc)) from None
def list_nodes(self, filter_str=None, format_str=None):
@@ -19,6 +175,10 @@ class NodeService(BaseService):
flags = re.IGNORECASE if not case_sensitive else 0
nodes = [n for n in nodes if re.search(filter_str, n, flags)]
ctx_svc = self._get_context_service()
if ctx_svc is not None and not ctx_svc.is_default_all_context():
nodes = ctx_svc.filter_node_list(result=nodes)
if not format_str:
return nodes
@@ -75,17 +235,21 @@ class NodeService(BaseService):
else:
flags = re.IGNORECASE if not case_sensitive else 0
folders = [f for f in folders if re.search(filter_str, f, flags)]
ctx_svc = self._get_context_service()
if ctx_svc is not None and not ctx_svc.is_default_all_context():
folders = ctx_svc.filter_node_list(result=folders)
return folders
def get_node_details(self, unique_id):
"""Return full configuration dictionary for a specific node."""
self._ensure_in_active_context(unique_id, is_folder=False)
try:
details = self.config.getitem(unique_id)
if not details:
raise NodeNotFoundError(f"Node '{unique_id}' not found.")
return details
except (KeyError, TypeError):
raise NodeNotFoundError(f"Node '{unique_id}' not found.")
raise NodeNotFoundError(f"Node '{unique_id}' not found.") from None
def explode_unique(self, unique_id):
"""Explode a unique ID into a dictionary of its parts."""
@@ -96,25 +260,54 @@ class NodeService(BaseService):
self.config._generate_nodes_cache(nodes=nodes, folders=folders, profiles=profiles)
def validate_parent_folder(self, unique_id, is_folder=False):
"""Check if parent folder exists for a given node unique ID."""
if is_folder:
uniques = self.config._explode_unique(unique_id)
if uniques and "subfolder" in uniques and "folder" in uniques:
parent_folder = f"@{uniques['folder']}"
if parent_folder not in self.config._getallfolders():
"""Check if unique_id is in active context and its parent folder exists."""
parent_folder = self._derive_parent_folder(self.config, unique_id, is_folder=is_folder)
ctx_svc = self._get_context_service()
if ctx_svc is not None:
ctx_svc.validate_in_active_context(unique_id, is_folder=is_folder)
if parent_folder is not None and not self._folder_exists(
parent_folder, self._get_raw_folders(), self.config
):
raise NodeNotFoundError(f"Folder '{parent_folder}' not found.")
else:
node_folder = unique_id.partition("@")[2]
if node_folder:
parent_folder = f"@{node_folder}"
if parent_folder not in self.config._getallfolders():
raise NodeNotFoundError(f"Folder '{parent_folder}' not found.")
def add_node(self, unique_id, data, is_folder=False):
"""Logic for adding a new node or folder to configuration."""
if not is_folder:
self._validate_node_name(unique_id)
self._validate_node_payload(data)
self.validate_parent_folder(unique_id, is_folder=is_folder)
if _supports_transactional_update(self.config):
def _mut(cfg):
all_nodes = cfg._getallnodes_unlocked()
all_folders = cfg._getallfolders_unlocked()
parent_folder = self._derive_parent_folder(cfg, unique_id, is_folder=is_folder)
ctx_svc = self._get_context_service(cfg)
if ctx_svc is not None:
ctx_svc.validate_in_active_context(unique_id, is_folder=is_folder)
if parent_folder is not None and not self._folder_exists(
parent_folder, all_folders, cfg
):
raise NodeNotFoundError(f"Folder '{parent_folder}' not found.")
if is_folder:
if unique_id in all_folders:
raise NodeAlreadyExistsError(f"Folder '{unique_id}' already exists.")
uniques = cfg._explode_unique_unlocked(unique_id)
if not uniques:
raise InvalidConfigurationError(f"Invalid folder name '{unique_id}'.")
cfg._folder_add_unlocked(**uniques)
else:
if unique_id in all_nodes:
raise NodeAlreadyExistsError(f"Node '{unique_id}' already exists.")
payload = deepcopy(data)
if "id" not in payload:
uniques = cfg._explode_unique_unlocked(unique_id)
if uniques and "id" in uniques:
payload["id"] = uniques["id"]
cfg._connections_add_unlocked(**payload)
self.config.transactional_update(_mut)
return
all_nodes = self.config._getallnodes()
all_folders = self.config._getallfolders()
@@ -126,7 +319,6 @@ class NodeService(BaseService):
if not uniques:
raise InvalidConfigurationError(f"Invalid folder name '{unique_id}'.")
# Check if parent folder exists when creating a subfolder
if "subfolder" in uniques:
self.validate_parent_folder(unique_id, is_folder=True)
@@ -136,37 +328,75 @@ class NodeService(BaseService):
if unique_id in all_nodes:
raise NodeAlreadyExistsError(f"Node '{unique_id}' already exists.")
# Check if parent folder exists when creating a node in a folder
self.validate_parent_folder(unique_id)
# Ensure 'id' is in data for config._connections_add
if "id" not in data:
payload = deepcopy(data)
if "id" not in payload:
uniques = self.config._explode_unique(unique_id)
if uniques and "id" in uniques:
data["id"] = uniques["id"]
payload["id"] = uniques["id"]
self.config._connections_add(**data)
self.config._connections_add(**payload)
self.config._saveconfig(self.config.file)
def _build_update_payload(self, unique_id, data, existing, uniques):
if isinstance(existing, dict):
merged = deepcopy(existing)
if isinstance(uniques, dict):
merged.update({k: v for k, v in uniques.items() if v})
merged.update(deepcopy(data))
filtered = {k: v for k, v in merged.items() if k in self._CONNECTION_FIELDS}
else:
filtered = deepcopy(data)
if "id" not in filtered and isinstance(uniques, dict) and "id" in uniques:
filtered["id"] = uniques["id"]
self._validate_node_payload(filtered)
return filtered
def update_node(self, unique_id, data, save=True):
"""Explicitly update an existing node."""
self._ensure_in_active_context(unique_id, is_folder=False)
if _supports_transactional_update(self.config) and save:
def _mut(cfg):
all_nodes = cfg._getallnodes_unlocked()
if unique_id not in all_nodes:
raise NodeNotFoundError(f"Node '{unique_id}' not found.")
uniques = cfg._explode_unique_unlocked(unique_id)
existing = cfg.getitem_unlocked(unique_id, extract=False)
payload = self._build_update_payload(unique_id, data, existing, uniques)
cfg._connections_add_unlocked(**payload)
self.config.transactional_update(_mut)
return
all_nodes = self.config._getallnodes()
if unique_id not in all_nodes:
raise NodeNotFoundError(f"Node '{unique_id}' not found.")
# Ensure 'id' is in data for config._connections_add
if "id" not in data:
uniques = self.config._explode_unique(unique_id)
if uniques:
data["id"] = uniques["id"]
# config._connections_add actually handles updates if ID exists correctly
self.config._connections_add(**data)
existing = self.config.getitem(unique_id, extract=False)
payload = self._build_update_payload(unique_id, data, existing, uniques)
self.config._connections_add(**payload)
if save:
self.config._saveconfig(self.config.file)
def delete_node(self, unique_id, is_folder=False, save=True):
"""Logic for deleting a node or folder."""
self._ensure_in_active_context(unique_id, is_folder=is_folder)
if _supports_transactional_update(self.config) and save:
def _mut(cfg):
uniques = cfg._explode_unique_unlocked(unique_id)
if not uniques:
label = "Folder" if is_folder else "Node"
raise NodeNotFoundError(f"{label} '{unique_id}' not found or invalid.")
if is_folder:
cfg._folder_del_unlocked(**uniques)
else:
cfg._connections_del_unlocked(**uniques)
self.config.transactional_update(_mut)
return
if is_folder:
uniques = self.config._explode_unique(unique_id)
if not uniques:
@@ -183,17 +413,35 @@ class NodeService(BaseService):
def connect_node(self, unique_id, sftp=False, debug=False, logger=None):
"""Interact with a node directly."""
self._ensure_in_active_context(unique_id, is_folder=False)
from connpy.core import node
from ..auth_utils import resolve_fallback_entries
from .profile_service import ProfileService
node_data = self.config.getitem(unique_id, extract=False)
if not node_data:
raise NodeNotFoundError(f"Node '{unique_id}' not found.")
# Resolve profiles
profile_service = ProfileService(self.config)
resolved_data = profile_service.resolve_node_data(node_data)
node_params = inspect.signature(node.__init__).parameters
if "fallback" not in node_params and "fallback" in resolved_data:
raw_fb = resolved_data.pop("fallback", None)
if raw_fb not in (None, "", []):
resolved_fb = resolve_fallback_entries(raw_fb, self.config.profiles)
if resolved_fb:
tags_val = resolved_data.get("tags", "")
if isinstance(tags_val, dict):
tags_copy = deepcopy(tags_val)
tags_copy["_fallback"] = resolved_fb
resolved_data["tags"] = tags_copy
elif tags_val in ("", None):
resolved_data["tags"] = {"_fallback": resolved_fb}
else:
resolved_data["tags"] = {"_raw": tags_val, "_fallback": resolved_fb}
n = node(unique_id, **resolved_data, config=self.config)
if sftp:
n.protocol = "sftp"
@@ -202,7 +450,48 @@ class NodeService(BaseService):
def move_node(self, src_id, dst_id, copy=False):
"""Move or copy a node."""
self._ensure_in_active_context(
src_id, is_folder=False, not_found_msg=f"Source node '{src_id}' not found."
)
self._validate_node_name(dst_id)
self.validate_parent_folder(dst_id, is_folder=False)
if _supports_transactional_update(self.config):
def _mut(cfg):
ctx_svc = self._get_context_service(cfg)
if (
ctx_svc is not None
and not ctx_svc.is_default_all_context()
and not ctx_svc.matches_active_context(src_id)
):
raise NodeNotFoundError(f"Source node '{src_id}' not found.")
parent_folder = self._derive_parent_folder(cfg, dst_id, is_folder=False)
if ctx_svc is not None:
ctx_svc.validate_in_active_context(dst_id, is_folder=False)
if parent_folder is not None and not self._folder_exists(
parent_folder, cfg._getallfolders_unlocked(), cfg
):
raise NodeNotFoundError(f"Folder '{parent_folder}' not found.")
try:
node_data = cfg.getitem_unlocked(src_id)
except (KeyError, TypeError):
node_data = None
if not node_data:
raise NodeNotFoundError(f"Source node '{src_id}' not found.")
if dst_id in cfg._getallnodes_unlocked():
raise NodeAlreadyExistsError(f"Destination node '{dst_id}' already exists.")
new_uniques = cfg._explode_unique_unlocked(dst_id)
if not new_uniques:
raise InvalidConfigurationError(f"Invalid destination format '{dst_id}'.")
new_node_data = deepcopy(node_data)
new_node_data.update(new_uniques)
cfg._connections_add_unlocked(**new_node_data)
if not copy:
src_uniques = cfg._explode_unique_unlocked(src_id)
cfg._connections_del_unlocked(**src_uniques)
self.config.transactional_update(_mut)
return
node_data = self.config.getitem(src_id)
if not node_data:
@@ -228,6 +517,55 @@ class NodeService(BaseService):
def bulk_add(self, ids, hosts, common_data):
"""Add multiple nodes with shared common configuration."""
self._validate_node_payload(common_data)
for uid in ids:
self.validate_parent_folder(uid, is_folder=False)
if _supports_transactional_update(self.config):
def _build_entries(cfg):
all_nodes = cfg._getallnodes_unlocked()
all_folders = cfg._getallfolders_unlocked()
ctx_svc = self._get_context_service(cfg)
planned = []
for i, uid in enumerate(ids):
parent_folder = self._derive_parent_folder(cfg, uid, is_folder=False)
if ctx_svc is not None:
ctx_svc.validate_in_active_context(uid, is_folder=False)
if parent_folder is not None and not self._folder_exists(
parent_folder, all_folders, cfg
):
raise NodeNotFoundError(f"Folder '{parent_folder}' not found.")
if uid in all_nodes:
continue
try:
self._validate_node_name(uid)
except ReservedNameError:
continue
host = hosts[i] if i < len(hosts) else hosts[0]
uniques = cfg._explode_unique_unlocked(uid)
if not uniques:
continue
node_data = deepcopy(common_data)
node_data.pop("ids", None)
node_data.pop("location", None)
node_data.update(uniques)
node_data["host"] = host
node_data["type"] = "connection"
planned.append(node_data)
return planned
with self.config._mutation_lock:
if not _build_entries(self.config):
return 0
def _mut(cfg):
entries = _build_entries(cfg)
for node_data in entries:
cfg._connections_add_unlocked(**node_data)
return len(entries)
return self.config.transactional_update(_mut)
count = 0
all_nodes = self.config._getallnodes()
@@ -238,8 +576,6 @@ class NodeService(BaseService):
try:
self._validate_node_name(uid)
except ReservedNameError:
# For bulk, we might want to just skip or log.
# CLI caller will handle if it wants to be strict.
continue
host = hosts[i] if i < len(hosts) else hosts[0]
@@ -263,6 +599,21 @@ class NodeService(BaseService):
def full_replace(self, connections, profiles):
"""Replace all connections and profiles with new data."""
from ..auth_utils import SanitizedAuthError
if _supports_transactional_update(self.config):
def _mut(cfg):
cfg.connections.clear()
cfg.connections.update(deepcopy(connections))
cfg.profiles.clear()
cfg.profiles.update(deepcopy(profiles))
try:
self.config.transactional_update(_mut)
except (ValueError, TypeError, SanitizedAuthError) as exc:
raise InvalidConfigurationError(str(exc)) from None
return
self.config.connections = connections
self.config.profiles = profiles
self.config._saveconfig(self.config.file)
+224 -57
View File
@@ -1,64 +1,211 @@
from contextlib import suppress
from copy import deepcopy
from typing import ClassVar
from .base import BaseService
from .exceptions import ProfileNotFoundError, ProfileAlreadyExistsError, InvalidConfigurationError
from .exceptions import (
InvalidConfigurationError,
ProfileAlreadyExistsError,
ProfileNotFoundError,
)
MAX_PROFILE_DEPTH = 5
def _supports_transactional_update(cfg):
return (
cfg is not None
and not hasattr(cfg, "assert_called")
and callable(getattr(cfg, "transactional_update", None))
)
class ProfileService(BaseService):
"""Business logic for node profiles management."""
_ALLOWED_KEYS: ClassVar[frozenset[str]] = frozenset({
"host",
"options",
"logs",
"password",
"port",
"protocol",
"user",
"tags",
"jumphost",
"fallback",
})
@staticmethod
def _validate_profile_payload(data):
from ..auth_utils import (
SanitizedAuthError,
is_compose_expression,
validate_auth_budget_tags,
validate_compose_syntax,
validate_fallback_entries,
)
try:
if "tags" in data and data["tags"] not in ("", None):
validate_auth_budget_tags(data["tags"])
if "fallback" in data and data["fallback"] not in (None, "", []):
validate_fallback_entries(data["fallback"])
pw = data.get("password")
if is_compose_expression(pw):
validate_compose_syntax(pw)
elif isinstance(pw, (list, tuple)):
for item in pw:
if is_compose_expression(item):
validate_compose_syntax(item)
except (ValueError, TypeError, SanitizedAuthError) as exc:
raise InvalidConfigurationError(str(exc)) from None
def _get_profiles_snapshot(self):
lock = getattr(self.config, "_mutation_lock", None)
if lock is not None and not hasattr(self.config, "assert_called"):
with lock:
return deepcopy(self.config.profiles)
return deepcopy(self.config.profiles)
def list_profiles(self, filter_str=None):
"""List all profile names, optionally filtered."""
profiles = list(self.config.profiles.keys())
profiles = list(self._get_profiles_snapshot().keys())
case_sensitive = self.config.config.get("case", False)
if filter_str:
if not case_sensitive:
f_str = filter_str.lower()
return [p for p in profiles if f_str in p.lower()]
else:
return [p for p in profiles if filter_str in p]
return profiles
def get_profile(self, name, resolve=True):
def get_profile(self, name, resolve=True, *, _visited=frozenset(), _depth=0):
"""Get the profile dictionary, optionally resolved."""
profile = self.config.profiles.get(name)
if _depth >= MAX_PROFILE_DEPTH or name in _visited:
raise InvalidConfigurationError(
f"Circular or overly deep profile reference involving '{name}'."
)
profiles = self._get_profiles_snapshot()
profile = profiles.get(name)
if not profile:
raise ProfileNotFoundError(f"Profile '{name}' not found.")
if resolve:
return self.resolve_node_data(profile)
return self.resolve_node_data(
profile,
_visited=_visited | {name},
_depth=_depth + 1,
_profiles_snapshot=profiles,
)
return profile
def add_profile(self, name, data):
"""Add a new profile."""
filtered_data = {k: deepcopy(v) for k, v in data.items() if k in self._ALLOWED_KEYS}
self._validate_profile_payload(filtered_data)
if _supports_transactional_update(self.config):
def _mut(cfg):
if name in cfg.profiles:
raise ProfileAlreadyExistsError(f"Profile '{name}' already exists.")
cfg._profiles_add_unlocked(id=name, **filtered_data)
self.config.transactional_update(_mut)
return
if name in self.config.profiles:
raise ProfileAlreadyExistsError(f"Profile '{name}' already exists.")
# Filter data to match _profiles_add signature and ensure id is passed
allowed_keys = {"host", "options", "logs", "password", "port", "protocol", "user", "tags", "jumphost"}
filtered_data = {k: v for k, v in data.items() if k in allowed_keys}
self.config._profiles_add(id=name, **filtered_data)
self.config._saveconfig(self.config.file)
def resolve_node_data(self, node_data):
"""Resolve profile references (@profile) in node data and handle inheritance."""
resolved = node_data.copy()
def resolve_node_data(
self,
node_data,
*,
_visited=frozenset(),
_depth=0,
_profiles_snapshot=None,
):
"""Resolve profile references (@profile) in node data while preserving compose and scalar secrets."""
from ..auth_utils import is_compose_expression
# 1. Identify all referenced profiles to support inheritance
referenced_profiles = []
for value in resolved.values():
if isinstance(value, str) and value.startswith("@"):
referenced_profiles.append(value[1:])
elif isinstance(value, list):
if _depth > MAX_PROFILE_DEPTH:
raise InvalidConfigurationError("Profile inheritance exceeds maximum depth.")
profiles = (
_profiles_snapshot
if _profiles_snapshot is not None
else self._get_profiles_snapshot()
)
resolved = deepcopy(node_data)
auth_referenced_profiles = []
for key in ("user", "password"):
value = resolved.get(key)
if isinstance(value, str):
if not is_compose_expression(value) and value.startswith("@"):
ref = value[1:]
if key != "password" or ref in profiles:
auth_referenced_profiles.append(ref)
elif isinstance(value, (list, tuple)):
for item in value:
if isinstance(item, str) and item.startswith("@"):
referenced_profiles.append(item[1:])
if (
isinstance(item, str)
and not is_compose_expression(item)
and item.startswith("@")
):
ref = item[1:]
if key != "password" or ref in profiles:
auth_referenced_profiles.append(ref)
def _get_resolved_ref(prof_name):
if prof_name in _visited or _depth >= MAX_PROFILE_DEPTH:
raise InvalidConfigurationError(
f"Circular or overly deep profile reference involving '{prof_name}'."
)
prof_raw = profiles.get(prof_name)
if not prof_raw:
raise ProfileNotFoundError(f"Profile '{prof_name}' not found.")
return self.resolve_node_data(
prof_raw,
_visited=_visited | {prof_name},
_depth=_depth + 1,
_profiles_snapshot=profiles,
)
for key, value in list(resolved.items()):
if key == "fallback":
continue
if key == "password":
if isinstance(value, str):
if is_compose_expression(value):
continue
if value.startswith("@") and value[1:] in profiles:
prof_obj = _get_resolved_ref(value[1:])
resolved["password"] = prof_obj.get("password", "")
elif isinstance(value, list):
resolved_list = []
for item in value:
if (
isinstance(item, str)
and not is_compose_expression(item)
and item.startswith("@")
and item[1:] in profiles
):
with suppress(ProfileNotFoundError):
prof_obj = _get_resolved_ref(item[1:])
if "password" in prof_obj:
resolved_list.append(prof_obj["password"])
else:
resolved_list.append(item)
resolved["password"] = resolved_list
continue
# 2. Resolve explicit references
for key, value in resolved.items():
if isinstance(value, str) and value.startswith("@"):
profile_name = value[1:]
try:
profile = self.get_profile(profile_name, resolve=True)
profile = _get_resolved_ref(profile_name)
resolved[key] = profile.get(key, "")
except ProfileNotFoundError:
resolved[key] = ""
@@ -67,33 +214,29 @@ class ProfileService(BaseService):
for item in value:
if isinstance(item, str) and item.startswith("@"):
profile_name = item[1:]
try:
profile = self.get_profile(profile_name, resolve=True)
with suppress(ProfileNotFoundError):
profile = _get_resolved_ref(profile_name)
if "password" in profile:
resolved_list.append(profile["password"])
except ProfileNotFoundError:
pass
else:
resolved_list.append(item)
resolved[key] = resolved_list
# 3. Inheritance: Fill empty keys from the first referenced profile
if referenced_profiles:
base_profile_name = referenced_profiles[0]
try:
base_profile = self.get_profile(base_profile_name, resolve=True)
for key, value in base_profile.items():
# Fill if key is missing or empty
if key not in resolved or resolved[key] == "" or resolved[key] == [] or resolved[key] is None:
resolved[key] = value
except ProfileNotFoundError:
pass
dedup_auth_profiles = list(dict.fromkeys(auth_referenced_profiles))
for base_profile_name in dedup_auth_profiles:
with suppress(ProfileNotFoundError):
base_profile = _get_resolved_ref(base_profile_name)
for key in ("user", "password", "fallback"):
val = base_profile.get(key)
if (
key not in resolved or resolved[key] in ("", [], None)
) and val not in ("", [], None):
resolved[key] = deepcopy(val)
# 4. Handle default protocol
if resolved.get("protocol") == "" or resolved.get("protocol") is None:
if resolved.get("protocol") in ("", None):
try:
default_profile = self.get_profile("default", resolve=True)
resolved["protocol"] = default_profile.get("protocol", "ssh")
default_profile = _get_resolved_ref("default")
resolved["protocol"] = default_profile.get("protocol", "ssh") or "ssh"
except ProfileNotFoundError:
resolved["protocol"] = "ssh"
@@ -101,34 +244,58 @@ class ProfileService(BaseService):
def delete_profile(self, name):
"""Delete an existing profile, with safety checks."""
if name not in self.config.profiles:
if _supports_transactional_update(self.config):
def _mut(cfg):
if name not in cfg.profiles:
raise ProfileNotFoundError(f"Profile '{name}' not found.")
if name == "default":
raise InvalidConfigurationError("Cannot delete the 'default' profile.")
used_by = cfg._profileused_unlocked(name)
if used_by:
raise InvalidConfigurationError(
f"Profile '{name}' is used by nodes: {', '.join(used_by)}"
)
cfg._profiles_del_unlocked(id=name)
self.config.transactional_update(_mut)
return
if name not in self.config.profiles:
raise ProfileNotFoundError(f"Profile '{name}' not found.")
if name == "default":
raise InvalidConfigurationError("Cannot delete the 'default' profile.")
used_by = self.config._profileused(name)
if used_by:
# We return the list of nodes using it so the UI can inform the user
raise InvalidConfigurationError(f"Profile '{name}' is used by nodes: {', '.join(used_by)}")
raise InvalidConfigurationError(
f"Profile '{name}' is used by nodes: {', '.join(used_by)}"
)
self.config._profiles_del(id=name)
self.config._saveconfig(self.config.file)
def update_profile(self, name, data):
"""Update an existing profile."""
if _supports_transactional_update(self.config):
def _mut(cfg):
if name not in cfg.profiles:
raise ProfileNotFoundError(f"Profile '{name}' not found.")
updated_data = deepcopy(cfg.profiles[name])
updated_data.update(deepcopy(data))
filtered_data = {
k: v for k, v in updated_data.items() if k in self._ALLOWED_KEYS
}
self._validate_profile_payload(filtered_data)
cfg._profiles_add_unlocked(id=name, **filtered_data)
self.config.transactional_update(_mut)
return
if name not in self.config.profiles:
raise ProfileNotFoundError(f"Profile '{name}' not found.")
# Merge with existing data
existing = self.get_profile(name, resolve=False)
updated_data = existing.copy()
updated_data = deepcopy(existing)
updated_data.update(data)
# Filter data to match _profiles_add signature
allowed_keys = {"host", "options", "logs", "password", "port", "protocol", "user", "tags", "jumphost"}
filtered_data = {k: v for k, v in updated_data.items() if k in allowed_keys}
filtered_data = {k: v for k, v in updated_data.items() if k in self._ALLOWED_KEYS}
self._validate_profile_payload(filtered_data)
self.config._profiles_add(id=name, **filtered_data)
self.config._saveconfig(self.config.file)
+65 -18
View File
@@ -1,5 +1,9 @@
import inspect
from contextlib import suppress
from .exceptions import InvalidConfigurationError
class RemoteStub:
def __getattr__(self, name):
raise NotImplementedError(
@@ -7,19 +11,26 @@ class RemoteStub:
"Use local mode or wait for the gRPC implementation."
)
class ServiceProvider:
"""Dynamic service backend. Transparently provides local or remote services."""
def __init__(self, config, mode="local", remote_host=None):
def __init__(self, config, mode="local", remote_host=None, enforce_context: bool = True):
from ..auth_utils import AuthAttemptTracker
self.mode = mode
self.config = config
self.remote_host = remote_host
self.enforce_context = bool(enforce_context)
self.auth_tracker = AuthAttemptTracker()
self.nodes = None
self._system = None
self._execution = None
self._import_export = None
self._sync = None
self._users = None
self._ai = None
self.set_enforce_context(self.enforce_context)
if mode == "local":
self._init_local()
@@ -28,14 +39,26 @@ class ServiceProvider:
else:
raise ValueError(f"Unknown service mode: {mode}")
def _init_local(self):
from .node_service import NodeService
from .profile_service import ProfileService
from .config_service import ConfigService
from .plugin_service import PluginService
from .context_service import ContextService
def set_enforce_context(self, enforce_context: bool) -> None:
self.enforce_context = bool(enforce_context)
if self.config is not None and not hasattr(self.config, "assert_called"):
self.config._enforce_context = self.enforce_context
for svc in (
getattr(self, "nodes", None),
self._execution,
self._import_export,
):
if svc is not None and hasattr(svc, "enforce_context"):
svc.enforce_context = self.enforce_context
self.nodes = NodeService(self.config)
def _init_local(self):
from .config_service import ConfigService
from .context_service import ContextService
from .node_service import NodeService
from .plugin_service import PluginService
from .profile_service import ProfileService
self.nodes = NodeService(self.config, enforce_context=self.enforce_context)
self.profiles = ProfileService(self.config)
self.config_svc = ConfigService(self.config)
self.plugins = PluginService(self.config)
@@ -45,18 +68,28 @@ class ServiceProvider:
# Allow ConfigService to work locally so the user can revert the mode
from .config_service import ConfigService
from .context_service import ContextService
self.config_svc = ConfigService(self.config)
self.context = ContextService(self.config)
if not self.remote_host:
raise InvalidConfigurationError("Remote host must be specified in remote mode")
import grpc
import os
import grpc
from ..grpc_layer.stubs import (
NodeStub, ProfileStub, PluginStub, AIStub,
ExecutionStub, ImportExportStub, SystemStub,
ConfigStub, AuthClientInterceptor, AuthStub
AIStub,
AuthClientInterceptor,
AuthStub,
ConfigStub,
ExecutionStub,
ImportExportStub,
NodeStub,
PluginStub,
ProfileStub,
SystemStub,
)
def get_token():
@@ -65,11 +98,8 @@ class ServiceProvider:
return env_token
token_path = os.path.join(self.config.defaultdir, ".token")
if os.path.exists(token_path):
try:
with open(token_path, "r") as f:
with suppress(OSError, UnicodeDecodeError), open(token_path) as f:
return f.read().strip()
except Exception:
pass
return None
channel = grpc.insecure_channel(self.remote_host)
@@ -86,7 +116,12 @@ class ServiceProvider:
self.plugins = PluginStub(channel, remote_host=self.remote_host)
self.ai = AIStub(channel, remote_host=self.remote_host)
self.system = SystemStub(channel, remote_host=self.remote_host)
self.execution = ExecutionStub(channel, remote_host=self.remote_host)
self.execution = ExecutionStub(
channel,
remote_host=self.remote_host,
config=self.config,
node_stub=self.nodes,
)
self.import_export = ImportExportStub(channel, remote_host=self.remote_host)
self.auth = AuthStub(channel, remote_host=self.remote_host)
@@ -105,7 +140,13 @@ class ServiceProvider:
def execution(self):
if self._execution is None and self.mode == "local":
from .execution_service import ExecutionService
self._execution = ExecutionService(self.config)
params = inspect.signature(ExecutionService.__init__).parameters
kwargs = {}
if "attempt_tracker" in params:
kwargs["attempt_tracker"] = self.auth_tracker
if "enforce_context" in params:
kwargs["enforce_context"] = self.enforce_context
self._execution = ExecutionService(self.config, **kwargs)
return self._execution
@execution.setter
@@ -116,6 +157,12 @@ class ServiceProvider:
def import_export(self):
if self._import_export is None and self.mode == "local":
from .import_export_service import ImportExportService
params = inspect.signature(ImportExportService.__init__).parameters
if "enforce_context" in params:
self._import_export = ImportExportService(
self.config, enforce_context=self.enforce_context
)
else:
self._import_export = ImportExportService(self.config)
return self._import_export
File diff suppressed because it is too large. Load diff
+501
View File
@@ -0,0 +1,501 @@
"""Unit tests for AI bugfixes, MethodHook instance isolation, and MCP multi-tenancy.
Covering specification: spec-02-ai-and-hooks-bugfixes.
"""
import os
from unittest.mock import AsyncMock, MagicMock, patch
import pytest
from connpy.ai import PlaybookBuilderAgent, ai
from connpy.hooks import MethodHook
from connpy.mcp_client import MCPClientManager
# =========================================================================
# 1. AI Long-Term Memory Bugfix Tests
# =========================================================================
class TestAILongTermMemory:
"""Tests ensuring long_term_memory interpolation and updates work correctly."""
def test_long_term_memory_injected_without_template_literal(self, ai_config, mock_litellm):
"""Verify no double braces or unresolved '{self.long_term_memory' literals exist."""
myai = ai(ai_config)
assert "{self.long_term_memory" not in myai.engineer_system_prompt
assert "{{self.long_term_memory" not in myai.engineer_system_prompt
assert "{self.long_term_memory" not in myai.architect_system_prompt
assert "Network Context: Empty." in myai.engineer_system_prompt
assert "Network Context: Empty." in myai.architect_system_prompt
def test_manage_memory_tool_updates_prompts_immediately(self, ai_config, mock_litellm):
"""Verify calling manage_memory_tool immediately updates engineer and architect prompts."""
myai = ai(ai_config)
res = myai.manage_memory_tool("DC-East ASN is 65010", action="append")
assert "Memory updated successfully" in res
assert "DC-East ASN is 65010" in myai.long_term_memory
assert "DC-East ASN is 65010" in myai.engineer_system_prompt
assert "DC-East ASN is 65010" in myai.architect_system_prompt
assert "{self.long_term_memory" not in myai.engineer_system_prompt
def test_init_with_preexisting_memory_file(self, ai_config, mock_litellm):
"""Verify preexisting ai_memory.md content is loaded into prompts on startup."""
mem_file = os.path.join(ai_config.defaultdir, "ai_memory.md")
with open(mem_file, "w") as f:
f.write("Border routers running BGP EVPN")
myai = ai(ai_config)
assert "Border routers running BGP EVPN" in myai.long_term_memory
assert "Border routers running BGP EVPN" in myai.engineer_system_prompt
assert "Border routers running BGP EVPN" in myai.architect_system_prompt
# =========================================================================
# 2. AI has_architect and architect_auth Tests
# =========================================================================
class TestAIArchitectAuthAndTools:
"""Tests ensuring architect tools and authentication function with architect_auth."""
def test_engineer_tools_include_architect_when_auth_configured_without_key(self, ai_config, mock_litellm):
"""consult_architect & escalate_to_architect must be available with architect_auth even without key."""
ai_config.config["ai"]["architect_api_key"] = None
ai_config.config["ai"]["architect_auth"] = {"service_account": "sa@google.com"}
myai = ai(ai_config)
assert myai.architect_key is None
assert myai.has_architect is True
tools = myai._get_engineer_tools()
tool_names = [t["function"]["name"] for t in tools]
assert "consult_architect" in tool_names
assert "escalate_to_architect" in tool_names
def test_engineer_tools_include_architect_for_keyless_model(self, ai_config, mock_litellm):
"""Vertex AI / Ollama keyless models must have architect tools enabled."""
ai_config.config["ai"]["architect_api_key"] = None
ai_config.config["ai"]["architect_auth"] = {}
ai_config.config["ai"]["architect_model"] = "vertex_ai/gemini-1.5-pro"
myai = ai(ai_config)
assert myai.has_architect is True
tools = myai._get_engineer_tools()
tool_names = [t["function"]["name"] for t in tools]
assert "consult_architect" in tool_names
assert "escalate_to_architect" in tool_names
def test_consult_architect_passes_auth_kwargs_cleanly(self, ai_config):
"""consult_architect must pass dict(architect_auth) to litellm.completion."""
ai_config.config["ai"]["architect_api_key"] = None
ai_config.config["ai"]["architect_auth"] = {
"api_base": "https://custom-ai.internal/v1",
"custom_header": "test-val",
}
myai = ai(ai_config)
with patch("connpy.ai.completion") as mock_comp:
fake_resp = MagicMock()
fake_msg = MagicMock()
fake_msg.content = "Architect advice on spine-leaf"
fake_msg.tool_calls = None
fake_msg.role = "assistant"
fake_msg.model_dump.return_value = {"role": "assistant", "content": "Advice"}
fake_choice = MagicMock()
fake_choice.message = fake_msg
fake_resp.choices = [fake_choice]
fake_resp.usage = MagicMock()
fake_resp.usage.prompt_tokens = 10
fake_resp.usage.completion_tokens = 10
fake_resp.usage.total_tokens = 20
# Craft first completion call that instructs engineer to consult architect
tc = MagicMock()
tc.function.name = "consult_architect"
tc.function.arguments = '{"question": "How to design MTU?", "technical_summary": "9000 vs 1500"}'
eng_msg = MagicMock()
eng_msg.role = "assistant"
eng_msg.content = ""
eng_msg.tool_calls = [tc]
eng_msg.model_dump.return_value = {
"role": "assistant",
"content": None,
"tool_calls": [{
"id": "call_1",
"type": "function",
"function": {
"name": "consult_architect",
"arguments": '{"question": "How to design MTU?", "technical_summary": "9000 vs 1500"}',
},
}],
}
eng_choice = MagicMock()
eng_choice.message = eng_msg
eng_resp = MagicMock()
eng_resp.choices = [eng_choice]
eng_resp.usage = MagicMock()
eng_resp.usage.prompt_tokens = 5
eng_resp.usage.completion_tokens = 5
eng_resp.usage.total_tokens = 10
# Final response after consultation
final_msg = MagicMock()
final_msg.role = "assistant"
final_msg.content = "Here is the guidance."
final_msg.tool_calls = None
final_msg.model_dump.return_value = {"role": "assistant", "content": "Here is the guidance."}
final_choice = MagicMock()
final_choice.message = final_msg
final_resp = MagicMock()
final_resp.choices = [final_choice]
final_resp.usage = MagicMock()
final_resp.usage.prompt_tokens = 5
final_resp.usage.completion_tokens = 5
final_resp.usage.total_tokens = 10
mock_comp.side_effect = [eng_resp, fake_resp, final_resp]
res = myai.ask("Design MTU question", stream=False)
assert "guidance" in res["response"]
# Second call to completion was consult_architect
assert mock_comp.call_count >= 2
consult_call = mock_comp.call_args_list[1]
kwargs = consult_call.kwargs
assert kwargs.get("api_base") == "https://custom-ai.internal/v1"
assert kwargs.get("custom_header") == "test-val"
assert "api_key" not in kwargs
def test_consult_architect_injects_architect_key_if_missing_from_auth(self, ai_config):
"""If architect_key exists and 'api_key' is not in architect_auth, it must be added."""
ai_config.config["ai"]["architect_api_key"] = "arch-key-12345"
ai_config.config["ai"]["architect_auth"] = {"api_base": "https://custom-ai.internal/v1"}
myai = ai(ai_config)
with patch("connpy.ai.completion") as mock_comp:
fake_resp = MagicMock()
fake_msg = MagicMock()
fake_msg.content = "Advice"
fake_msg.tool_calls = None
fake_msg.role = "assistant"
fake_msg.model_dump.return_value = {"role": "assistant", "content": "Advice"}
fake_choice = MagicMock()
fake_choice.message = fake_msg
fake_resp.choices = [fake_choice]
fake_resp.usage = MagicMock()
fake_resp.usage.prompt_tokens = 10
fake_resp.usage.completion_tokens = 10
fake_resp.usage.total_tokens = 20
tc = MagicMock()
tc.function.name = "consult_architect"
tc.function.arguments = '{"question": "Q?", "technical_summary": "S"}'
eng_msg = MagicMock()
eng_msg.role = "assistant"
eng_msg.content = ""
eng_msg.tool_calls = [tc]
eng_msg.model_dump.return_value = {
"role": "assistant",
"content": None,
"tool_calls": [{
"id": "call_1",
"type": "function",
"function": {
"name": "consult_architect",
"arguments": '{"question": "Q?", "technical_summary": "S"}',
},
}],
}
eng_choice = MagicMock()
eng_choice.message = eng_msg
eng_resp = MagicMock()
eng_resp.choices = [eng_choice]
eng_resp.usage = MagicMock()
eng_resp.usage.prompt_tokens = 5
eng_resp.usage.completion_tokens = 5
eng_resp.usage.total_tokens = 10
final_msg = MagicMock()
final_msg.role = "assistant"
final_msg.content = "Done."
final_msg.tool_calls = None
final_msg.model_dump.return_value = {"role": "assistant", "content": "Done."}
final_choice = MagicMock()
final_choice.message = final_msg
final_resp = MagicMock()
final_resp.choices = [final_choice]
final_resp.usage = MagicMock()
final_resp.usage.prompt_tokens = 5
final_resp.usage.completion_tokens = 5
final_resp.usage.total_tokens = 10
mock_comp.side_effect = [eng_resp, fake_resp, final_resp]
myai.ask("Consult question", stream=False)
consult_call = mock_comp.call_args_list[1]
kwargs = consult_call.kwargs
assert kwargs.get("api_key") == "arch-key-12345"
assert kwargs.get("api_base") == "https://custom-ai.internal/v1"
# =========================================================================
# 3. PlaybookBuilderAgent Safe YAML Validation Tests
# =========================================================================
class TestPlaybookValidationSafeLoad:
"""Tests verifying PlaybookBuilderAgent.validate_playbook uses safe_load."""
def test_validate_playbook_rejects_malicious_yaml_tags(self, config):
"""Ensure arbitrary Python object deserialization payloads are safely rejected."""
agent = PlaybookBuilderAgent(config)
malicious_yaml = """
tasks:
- name: Exploit
command: !!python/object/apply:os.system ["touch /tmp/hacked_marker"]
"""
res = agent.validate_playbook(malicious_yaml)
assert res["valid"] is False
assert "YAML Syntax Error" in res["error"]
assert not os.path.exists("/tmp/hacked_marker")
def test_validate_playbook_accepts_valid_yaml(self, config):
"""Ensure safe YAML with tasks structure validates successfully."""
agent = PlaybookBuilderAgent(config)
valid_yaml = """
tasks:
- name: Check BGP
action: run
nodes: "router.*"
commands:
- show ip bgp summary
output: stdout
"""
res = agent.validate_playbook(valid_yaml)
assert res["valid"] is True
def test_validate_playbook_invalid_structures(self, config):
"""Ensure non-dict or missing 'tasks' YAML is rejected cleanly."""
agent = PlaybookBuilderAgent(config)
assert agent.validate_playbook("- not a dict\n- just a list")["valid"] is False
assert agent.validate_playbook("simple_scalar_string")["valid"] is False
assert agent.validate_playbook("other_root: 123")["valid"] is False
assert agent.validate_playbook("tasks: 'not a list'")["valid"] is False
# =========================================================================
# 4. MethodHook Instance Isolation and Deduplication Tests
# =========================================================================
class TestMethodHookIsolation:
"""Tests ensuring MethodHook instances are strictly isolated and deduplicated."""
def test_instance_hooks_do_not_contaminate_other_instances(self):
"""Registering a hook on inst1 must not run on inst2 or modify class hooks."""
class NetworkService:
@MethodHook
def run(self, cmd):
return f"executed:{cmd}"
inst1 = NetworkService()
inst2 = NetworkService()
def inst1_post_hook(*args, **kwargs):
return kwargs["result"] + " [audited-by-1]"
inst1.run.register_post_hook(inst1_post_hook)
assert inst1.run("show clock") == "executed:show clock [audited-by-1]"
assert inst2.run("show clock") == "executed:show clock"
assert NetworkService.run.post_hooks == []
def test_instance_hooks_deduplication(self):
"""Registering the same callable multiple times on an instance runs only once."""
class NetworkService:
@MethodHook
def run(self, cmd):
return f"res:{cmd}"
calls = []
def counting_hook(*args, **kwargs):
calls.append("called")
return kwargs["result"]
inst = NetworkService()
inst.run.register_post_hook(counting_hook)
inst.run.register_post_hook(counting_hook)
inst.run.register_post_hook(counting_hook)
res = inst.run("test")
assert res == "res:test"
assert calls == ["called"]
def test_class_level_and_instance_level_execution_order(self):
"""Class pre-hooks run before instance pre-hooks; class post-hooks run before instance post-hooks."""
class ExecutionDevice:
@MethodHook
def execute(self, val):
trace.append(f"base:{val}")
return val
trace = []
def class_pre(self, val):
trace.append("class_pre")
return (self, val), {}
def inst_pre(self, val):
trace.append("inst_pre")
return (self, val), {}
def class_post(*args, **kwargs):
trace.append("class_post")
return kwargs["result"]
def inst_post(*args, **kwargs):
trace.append("inst_post")
return kwargs["result"]
ExecutionDevice.execute.register_pre_hook(class_pre)
ExecutionDevice.execute.register_post_hook(class_post)
dev = ExecutionDevice()
dev.execute.register_pre_hook(inst_pre)
dev.execute.register_post_hook(inst_post)
dev.execute("ping")
assert trace == ["class_pre", "inst_pre", "base:ping", "class_post", "inst_post"]
def test_configfile_instance_isolation(self, config):
"""Verify two configfile instances do not leak hooks across instances."""
from connpy.configfile import configfile
conf2 = configfile(conf=config.file, key=config.key)
captured = []
def custom_hook(*args, **kwargs):
captured.append("c1_hook")
return kwargs["result"]
config._getallnodes.register_post_hook(custom_hook)
config._getallnodes(".*")
assert captured == ["c1_hook"]
# Call on second instance — hook must NOT execute
captured.clear()
conf2._getallnodes(".*")
assert captured == []
# =========================================================================
# 5. MCPClientManager Multi-Tenancy and Invalidation Tests
# =========================================================================
class DummyConfig:
"""Mock config object with config attribute and defaultdir."""
def __init__(self, defaultdir: str, mcp_servers: dict | None = None):
self.defaultdir = defaultdir
self.config = {
"ai": {
"mcp_servers": mcp_servers or {},
},
}
class TestMCPClientManagerMultiTenancy:
"""Tests ensuring MCPClientManager isolates tenants and invalidates cache properly."""
@pytest.fixture(autouse=True)
def reset_mcp_singletons(self):
"""Clean MCPClientManager instances before and after each test."""
MCPClientManager._instances.clear()
MCPClientManager._instance = None
yield
MCPClientManager._instances.clear()
MCPClientManager._instance = None
def test_mcp_manager_scoped_by_defaultdir(self):
"""Different defaultdir paths produce distinct isolated instances."""
cfg_tenant1 = DummyConfig("/var/connpy/tenant1")
cfg_tenant2 = DummyConfig("/var/connpy/tenant2")
mgr1 = MCPClientManager(cfg_tenant1)
mgr2 = MCPClientManager(cfg_tenant2)
assert mgr1 is not mgr2
assert mgr1.config is cfg_tenant1
assert mgr2.config is cfg_tenant2
def test_mcp_manager_same_scope_returns_same_instance_and_updates_config(self):
"""Same defaultdir reuses manager and updates self.config."""
cfg_a = DummyConfig("/var/connpy/shared")
cfg_b = DummyConfig("/var/connpy/shared")
mgr_a = MCPClientManager(cfg_a)
mgr_b = MCPClientManager(cfg_b)
assert mgr_a is mgr_b
assert mgr_a.config is cfg_b
def test_mcp_manager_scoped_by_id_when_no_defaultdir(self):
"""Configs without defaultdir are isolated by id(config)."""
cfg1 = object()
cfg2 = object()
mgr1 = MCPClientManager(cfg1)
mgr2 = MCPClientManager(cfg2)
assert mgr1 is not mgr2
@pytest.mark.asyncio
async def test_get_tools_invalidates_cache_on_config_change(self):
"""When server config changes in mcp_servers, tool_cache is invalidated."""
cfg = DummyConfig(
"/tmp/test_mcp_invalidation",
mcp_servers={"srv1": {"url": "http://127.0.0.1:8001/sse", "enabled": True}},
)
mgr = MCPClientManager(cfg)
mgr.tool_cache["srv1"] = [{"type": "function", "function": {"name": "old_tool"}}]
mock_stack = AsyncMock()
mgr.sessions["srv1"] = {"session": MagicMock(), "stack": mock_stack}
mgr._server_configs["srv1"] = {"url": "http://127.0.0.1:8001/sse", "enabled": True}
# Modify config to point to new URL
cfg.config["ai"]["mcp_servers"]["srv1"] = {"url": "http://127.0.0.1:8002/sse", "enabled": True}
new_session = MagicMock()
with patch.object(mgr, "_ensure_connected", new_callable=AsyncMock) as mock_conn, \
patch.object(mgr, "_fetch_tools_as_openai", new_callable=AsyncMock) as mock_fetch:
mock_conn.return_value = new_session
mock_fetch.return_value = [{"type": "function", "function": {"name": "new_tool"}}]
tools = await mgr.get_tools_for_llm()
# Ensure old stack was closed
mock_stack.aclose.assert_awaited_once()
# Ensure new tools returned
assert tools == [{"type": "function", "function": {"name": "new_tool"}}]
assert mgr.tool_cache["srv1"] == [{"type": "function", "function": {"name": "new_tool"}}]
await mgr.shutdown()
@pytest.mark.asyncio
async def test_shutdown_cleans_resources_and_resets(self):
"""shutdown() closes all stacks, clears caches and removes from _instances."""
cfg = DummyConfig("/tmp/test_mcp_shutdown")
mgr = MCPClientManager(cfg)
mock_stack = AsyncMock()
mgr.sessions["test"] = {"session": MagicMock(), "stack": mock_stack}
mgr.tool_cache["test"] = [{"dummy": 1}]
await mgr.shutdown()
mock_stack.aclose.assert_awaited_once()
assert mgr.sessions == {}
assert mgr.tool_cache == {}
assert os.path.abspath(cfg.defaultdir) not in MCPClientManager._instances
File diff suppressed because it is too large. Load diff
+15
View File
@@ -80,11 +80,22 @@ class TestTreeCompletions:
assert "--shell-command" in config_completions
assert "--shell-prompt" in config_completions
assert "--shell-os" in config_completions
assert "--max-auth-prompts" in config_completions
assert "--max-total-auth-attempts" in config_completions
assert "--max-hop-host-attempts" in config_completions
assert "--max-global-user-attempts" in config_completions
assert "--disable-fallback-on-insecure-hostkey" in config_completions
# Resolve when --engineer-auth is chosen in config
auth_comp = resolve_completion(["config", "--engineer-auth", ""], tree)
assert isinstance(auth_comp, list)
# Resolve boolean values for --disable-fallback-on-insecure-hostkey
assert resolve_completion(
["config", "--disable-fallback-on-insecure-hostkey", ""],
tree,
) == ["true", "false"]
# Loop back check:
# e.g., connpy config --engineer-auth some_val
# should loop back and resolve to config options
@@ -93,6 +104,10 @@ class TestTreeCompletions:
assert "--engineer-auth" in loop_back_comp
assert "--shell-command" in loop_back_comp
auth_budget_loop_back = resolve_completion(["config", "--max-auth-prompts", "3", ""], tree)
assert "--max-total-auth-attempts" in auth_budget_loop_back
assert "--disable-fallback-on-insecure-hostkey" in auth_budget_loop_back
def test_shell_completions(self):
from connpy.completion import _build_tree, resolve_completion
tree = _build_tree([], [], [], {}, "/tmp")
+413
View File
@@ -0,0 +1,413 @@
"""Tests for ConnpyMCPServer (spec-03-mcp-server)."""
import sys
from unittest.mock import MagicMock, patch
import pytest
from connpy.mcp_server import ConnpyMCPServer
from connpy.services.provider import ServiceProvider
@pytest.fixture
def services(populated_config):
"""Provide ServiceProvider initialized with sample connections/profiles."""
return ServiceProvider(populated_config)
@pytest.fixture
def mcp_server(populated_config, services):
"""Provide ConnpyMCPServer in untrusted mode."""
return ConnpyMCPServer(config=populated_config, services=services, trust=False)
@pytest.fixture
def trusted_mcp_server(populated_config, services):
"""Provide ConnpyMCPServer in trusted mode."""
return ConnpyMCPServer(config=populated_config, services=services, trust=True)
# =============================================================================
# Level 1 (Read-Only) Tools & Password Masking Tests
# =============================================================================
class TestLevel1Tools:
def test_list_nodes_all(self, mcp_server):
nodes = mcp_server.list_nodes()
assert isinstance(nodes, list)
assert "router1" in nodes
assert "server1@office" in nodes
assert "db1@datacenter@office" in nodes
def test_list_nodes_filtered(self, mcp_server):
nodes = mcp_server.list_nodes(filter_pattern="router.*")
assert "router1" in nodes
assert "server1@office" not in nodes
def test_list_nodes_formatted(self, mcp_server):
fmt = "{name} - {location}"
formatted = mcp_server.list_nodes(filter_pattern="router1", format_str=fmt)
assert len(formatted) == 1
assert formatted[0] == "router1 - root"
def test_list_folders(self, mcp_server):
folders = mcp_server.list_folders()
assert "@office" in folders
assert "@datacenter@office" in folders
def test_list_folders_filtered(self, mcp_server):
folders = mcp_server.list_folders(filter_pattern="datacenter")
assert "@datacenter@office" in folders
assert "@office" not in folders
def test_get_node_info_masks_password(self, mcp_server):
info = mcp_server.get_node_info("router1")
assert isinstance(info, dict)
assert info["host"] == "10.0.0.1"
assert info["user"] == "admin"
# CRITICAL requirement: ALWAYS mask password/passphrase/secret with "***"
assert info["password"] == "***"
assert "pass1" not in str(info)
def test_get_node_info_masks_nested_secrets(self, mcp_server):
test_data = {
"password": "plain_password",
"secret": "enable_secret",
"passphrase": "ssh_passphrase",
"api_token": "bearer_token",
"nested": {
"admin_password": "secret_nested",
"normal_key": "safe_value",
},
}
masked = mcp_server._mask_secrets(test_data)
assert masked["password"] == "***"
assert masked["secret"] == "***"
assert masked["passphrase"] == "***"
assert masked["api_token"] == "***"
assert masked["nested"]["admin_password"] == "***"
assert masked["nested"]["normal_key"] == "safe_value"
def test_get_node_info_not_found(self, mcp_server):
res = mcp_server.get_node_info("non_existent_node")
assert "error" in res
assert "not found" in res["error"].lower()
def test_list_profiles(self, mcp_server):
profiles = mcp_server.list_profiles()
assert "default" in profiles
assert "office-user" in profiles
def test_list_profiles_filtered(self, mcp_server):
profiles = mcp_server.list_profiles(filter_pattern="office")
assert "office-user" in profiles
assert "default" not in profiles
def test_get_active_context(self, mcp_server):
ctx = mcp_server.get_active_context()
assert isinstance(ctx, dict)
assert "context" in ctx
assert ctx["context"] == "all"
assert "patterns" in ctx
assert ".*" in ctx["patterns"]
# =============================================================================
# Level 2 Security & Execution Tests (is_safe_command, run, test, playbooks)
# =============================================================================
class TestLevel2Tools:
def test_is_safe_command_allowed_patterns(self, mcp_server):
safe_samples = [
"show version",
"show ip route",
"ls -la /var/log",
"cat /etc/hosts",
"ip address",
"pwd",
"hostname",
"uname -a",
"df -h",
"free -m",
"ps aux",
"ping -c 4 8.8.8.8",
"traceroute 1.1.1.1",
"whois example.com",
"kubectl get pods -n default",
"systemctl status nginx",
"journalctl -u ssh -n 50",
]
for cmd in safe_samples:
assert mcp_server.is_safe_command(cmd) is True, f"Expected safe: {cmd}"
def test_is_safe_command_blocks_unsafe_in_untrusted_mode(self, mcp_server):
unsafe_samples = [
"reboot",
"rm -rf /",
"shutdown -h now",
"configure terminal",
"set system host-name bad",
"echo hacked",
"dd if=/dev/zero of=/dev/sda",
]
for cmd in unsafe_samples:
assert mcp_server.is_safe_command(cmd) is False, f"Expected unsafe: {cmd}"
def test_is_safe_command_blocks_injection_characters(self, mcp_server):
injection_samples = [
"show version; rm -rf /",
"show version && reboot",
"show version || reboot",
"cat /etc/hosts | grep root",
"show `whoami`",
"cat $(id)",
"echo test > /tmp/bad",
"cat < /etc/shadow",
"show ip\nreboot",
"show ip\rreboot",
"show version & reboot",
]
for cmd in injection_samples:
assert not mcp_server.is_safe_command(cmd), f"Injection allowed: {cmd}"
def test_is_safe_command_allows_all_in_trusted_mode(self, trusted_mcp_server):
commands = [
"reboot",
"rm -rf /",
"configure terminal",
"show version; reboot",
"cat /etc/hosts | grep foo",
]
for cmd in commands:
assert trusted_mcp_server.is_safe_command(cmd) is True
def test_run_commands_blocks_unsafe_when_untrusted(self, mcp_server):
res = mcp_server.run_commands("router1", ["show version", "reboot"])
assert res["success"] is False
assert "Permission denied" in res["error"]
assert "reboot" in res["error"]
def test_run_commands_executes_when_trusted(self, trusted_mcp_server):
exec_svc = trusted_mcp_server.services.execution
with patch.object(exec_svc, "run_commands") as mock_exec:
mock_exec.return_value = {"router1": {"output": "ok", "status": 0}}
res = trusted_mcp_server.run_commands("router1", ["reboot"])
assert res["success"] is True
assert res["results"]["router1"]["output"] == "ok"
mock_exec.assert_called_once()
def test_test_commands_blocks_unsafe_when_untrusted(self, mcp_server):
res = mcp_server.test_commands("router1", ["rm -rf /"], expected=["root"])
assert res["success"] is False
assert "Permission denied" in res["error"]
def test_test_commands_executes_when_trusted(self, trusted_mcp_server):
exec_svc = trusted_mcp_server.services.execution
with patch.object(exec_svc, "test_commands") as mock_test:
mock_test.return_value = {"router1": {"rm -rf /": True}}
res = trusted_mcp_server.test_commands(
"router1", ["rm -rf /"], expected=["root"]
)
assert res["success"] is True
assert res["results"]["router1"]["rm -rf /"] is True
def test_validate_playbook_valid(self, mcp_server):
valid_yaml = """
tasks:
- name: Check Status
action: run
nodes: router1
commands:
- show version
output: stdout
- name: Assert Interface
action: test
nodes: router1
commands:
- show ip int brief
expected:
- GigabitEthernet0/0
output: stdout
"""
res = mcp_server.validate_playbook(valid_yaml)
assert res["valid"] is True
def test_validate_playbook_invalid(self, mcp_server):
# Missing root tasks key
res1 = mcp_server.validate_playbook("name: invalid")
assert res1["valid"] is False
assert "tasks" in res1["error"]
# Missing expected in test action
invalid_yaml = """
tasks:
- name: Bad Test
action: test
nodes: router1
commands:
- show version
output: stdout
"""
res2 = mcp_server.validate_playbook(invalid_yaml)
assert res2["valid"] is False
assert "expected" in res2["error"]
def test_run_playbook_blocks_unsafe_commands_when_untrusted(self, mcp_server):
unsafe_playbook = """
tasks:
- name: Reboot Device
action: run
nodes: router1
commands:
- reboot
output: stdout
"""
res = mcp_server.run_playbook(unsafe_playbook)
assert res["success"] is False
assert "Permission denied" in res["error"]
assert "reboot" in res["error"]
def test_run_playbook_executes_when_trusted(self, trusted_mcp_server):
playbook = """
tasks:
- name: Safe Task
action: run
nodes: router1
commands:
- reboot
output: stdout
"""
exec_svc = trusted_mcp_server.services.execution
with patch.object(exec_svc, "run_commands") as mock_exec:
mock_exec.return_value = {"router1": {"output": "ok", "status": 0}}
res = trusted_mcp_server.run_playbook(playbook)
assert res["success"] is True
assert "Safe Task" in res["results"]
# =============================================================================
# Level 3 (CRUD) Inventory Mutation Tests
# =============================================================================
class TestLevel3CRUD:
def test_add_node_blocked_when_untrusted(self, mcp_server):
res = mcp_server.add_node("router99", {"host": "10.0.0.99"})
assert res["success"] is False
assert "Permission denied" in res["error"]
def test_add_node_allowed_when_trusted(self, trusted_mcp_server):
data = {"host": "10.0.0.99", "protocol": "ssh"}
res = trusted_mcp_server.add_node("router99", data)
assert res["success"] is True
nodes = trusted_mcp_server.list_nodes()
assert "router99" in nodes
def test_update_node_blocked_when_untrusted(self, mcp_server):
res = mcp_server.update_node("router1", {"host": "10.0.0.254"})
assert res["success"] is False
assert "Permission denied" in res["error"]
def test_update_node_allowed_when_trusted(self, trusted_mcp_server):
res = trusted_mcp_server.update_node("router1", {"host": "10.0.0.254"})
assert res["success"] is True
info = trusted_mcp_server.get_node_info("router1")
assert info["host"] == "10.0.0.254"
def test_delete_node_blocked_when_untrusted(self, mcp_server):
res = mcp_server.delete_node("router1")
assert res["success"] is False
assert "Permission denied" in res["error"]
def test_delete_node_allowed_when_trusted(self, trusted_mcp_server):
res = trusted_mcp_server.delete_node("router1")
assert res["success"] is True
nodes = trusted_mcp_server.list_nodes()
assert "router1" not in nodes
# =============================================================================
# Transport & Logging Redirection Tests
# =============================================================================
class TestTransportAndRedirection:
def test_stdio_redirects_stdout_and_calls_run(self, mcp_server):
with patch.object(mcp_server.server, "run") as mock_run, \
patch("connpy.printer.set_thread_stream") as mock_set_stream:
mcp_server.run(transport="stdio")
mock_set_stream.assert_called_once_with(sys.stderr)
mock_run.assert_called_once_with(transport="stdio")
def test_sse_passes_host_and_port(self, mcp_server):
with patch.object(mcp_server.server, "run") as mock_run:
mcp_server.run(transport="sse", host="0.0.0.0", port=9090)
mock_run.assert_called_once_with(transport="sse", host="0.0.0.0", port=9090)
# =============================================================================
# Async MCP Protocol Tool Invocation Tests
# =============================================================================
class TestMCPProtocolAsyncCall:
@pytest.mark.asyncio
async def test_async_call_tool_list_nodes(self, mcp_server):
res = await mcp_server.call_tool("list_nodes", {"filter_pattern": "router1"})
assert not res.is_error
text = res.content[0].text
assert "router1" in text
@pytest.mark.asyncio
async def test_async_call_tool_get_node_info_masks_password(self, mcp_server):
res = await mcp_server.call_tool("get_node_info", {"node_name": "router1"})
assert not res.is_error
text = res.content[0].text
assert '"password": "***"' in text
assert "pass1" not in text
# =============================================================================
# CLI Dispatch & Completion Tests
# =============================================================================
class TestCLIDispatchAndCompletion:
def test_ai_handler_configure_mcp_serve_dispatch(self, populated_config, services):
from connpy.cli.ai_handler import AIHandler
class DummyApp:
def __init__(self, conf, svcs):
self.config = conf
self.services = svcs
app = DummyApp(populated_config, services)
handler = AIHandler(app)
args = MagicMock()
args.mcp = ["serve", "--sse", "--port", "9191", "--trust"]
args.trust = False
with patch("connpy.mcp_server.ConnpyMCPServer.run") as mock_run:
handler.configure_mcp(args)
mock_run.assert_called_once_with(
transport="sse", host="127.0.0.1", port=9191
)
def test_completion_contains_serve_flags(self):
from connpy.completion import _build_tree, resolve_completion
tree = _build_tree([], [], [], {}, "/tmp")
# Verify tree hierarchy
assert "serve" in tree["ai"]["--mcp"]
serve_sub = tree["ai"]["--mcp"]["serve"]
assert "--sse" in serve_sub
assert "--port" in serve_sub
assert "--trust" in serve_sub
# Verify completions resolution
comp_base = resolve_completion(["ai", "--mcp", "serve", ""], tree)
assert "--sse" in comp_base
assert "--port" in comp_base
assert "--trust" in comp_base
# After --sse, remaining flags are suggested
comp_after_sse = resolve_completion(["ai", "--mcp", "serve", "--sse", ""], tree)
assert "--port" in comp_after_sse
assert "--trust" in comp_after_sse
+222
View File
@@ -0,0 +1,222 @@
"""Tests for YAML safe deserialization and connection options sanitization with allow_unsafe_options."""
from unittest.mock import MagicMock, patch
import pytest
from connpy.cli.run_handler import RunHandler
from connpy.core import node
from connpy.grpc_layer.server import ImportExportServicer
from connpy.services.exceptions import InvalidConfigurationError
from connpy.services.import_export_service import ImportExportService
# =========================================================================
# 1. Tests de Deserialización YAML Segura
# =========================================================================
class TestYamlSafeDeserialization:
def test_import_export_service_rejects_unsafe_yaml(self, config, tmp_path):
"""YAML with !!python/object/apply must fail with InvalidConfigurationError and never execute."""
malicious_yaml = tmp_path / "malicious.yaml"
malicious_yaml.write_text('!!python/object/apply:os.system ["echo pwned"]')
service = ImportExportService(config)
with patch("os.system") as mock_system:
with pytest.raises(InvalidConfigurationError):
service.import_from_file(str(malicious_yaml))
mock_system.assert_not_called()
def test_grpc_import_from_file_rejects_unsafe_yaml(self):
"""gRPC import with !!python/object/apply must abort via context and never execute."""
payload = '---YAML---\n!!python/object/apply:os.system ["echo pwned"]'
request = MagicMock()
request.value = payload
context = MagicMock()
provider = MagicMock()
servicer = ImportExportServicer(provider)
with patch("os.system") as mock_system:
servicer.import_from_file(request, context)
context.abort.assert_called_once()
mock_system.assert_not_called()
def test_run_handler_yaml_run_rejects_unsafe_yaml(self, tmp_path):
"""RunHandler.yaml_run with !!python/object/apply exits cleanly with code 10 and never executes."""
malicious_playbook = tmp_path / "bad_playbook.yaml"
malicious_playbook.write_text(
'tasks:\n - name: exploit\n action: run\n nodes: all\n commands:\n - !!python/object/apply:os.system ["echo pwned"]\n'
)
app = MagicMock()
handler = RunHandler(app)
args = MagicMock()
args.data = [str(malicious_playbook)]
args.preflight_ai = False
args.analyze = None
with patch("os.system") as mock_system:
with pytest.raises(SystemExit) as exc_info:
handler.yaml_run(args)
assert exc_info.value.code == 10
mock_system.assert_not_called()
def test_run_handler_yaml_run_rejects_non_dict_yaml(self, tmp_path):
"""RunHandler.yaml_run with empty or non-dict YAML exits cleanly with code 10."""
non_dict_playbook = tmp_path / "list_playbook.yaml"
non_dict_playbook.write_text("- item1\n- item2\n")
app = MagicMock()
handler = RunHandler(app)
args = MagicMock()
args.data = [str(non_dict_playbook)]
args.preflight_ai = False
args.analyze = None
with pytest.raises(SystemExit) as exc_info:
handler.yaml_run(args)
assert exc_info.value.code == 10
# =========================================================================
# 2. Tests Negativos de Inyección en Protocolos (allow_unsafe_options=False)
# =========================================================================
class TestProtocolInjectionNegative:
@pytest.mark.parametrize("proto", ["ssh", "sftp", "telnet", "kubectl", "docker", "ssm"])
def test_host_starting_with_dash_raises_value_error(self, proto):
"""Host starting with '-' raises ValueError across all protocols."""
n = node("bad_node", "-oProxyCommand=id", protocol=proto)
with pytest.raises(ValueError, match="Host cannot start with '-'"):
n._get_cmd()
@pytest.mark.parametrize("proto", ["ssh", "sftp", "telnet", "kubectl", "docker", "ssm"])
def test_host_with_unsafe_characters_raises_value_error(self, proto):
"""Host containing whitespace, control characters or shell metacharacters raises ValueError."""
for bad_host in ["host; id", "host | id", "host`id`", "host$id", "host&id"]:
n = node("bad_node", bad_host, protocol=proto)
with pytest.raises(ValueError, match="Invalid characters in host"):
n._get_cmd()
@pytest.mark.parametrize("proto", ["ssh", "sftp", "telnet", "kubectl", "docker", "ssm"])
def test_user_starting_with_dash_raises_value_error(self, proto):
"""User starting with '-' raises ValueError across all protocols."""
n = node("n1", "10.0.0.1", user="-oProxyCommand=id", protocol=proto)
with pytest.raises(ValueError, match="User cannot start with '-'"):
n._get_cmd()
@pytest.mark.parametrize("proto", ["ssh", "sftp", "telnet", "kubectl", "docker", "ssm"])
def test_user_with_unsafe_characters_raises_value_error(self, proto):
"""User containing shell metacharacters raises ValueError across all protocols."""
for bad_user in ["; id", "user | id", "user`id`", "user$id", "user&id"]:
n = node("n1", "10.0.0.1", user=bad_user, protocol=proto)
with pytest.raises(ValueError, match="Invalid characters in user"):
n._get_cmd()
@pytest.mark.parametrize("proto", ["ssh", "sftp", "telnet"])
def test_invalid_port_raises_value_error(self, proto):
"""Invalid port in SSH/SFTP/Telnet raises ValueError."""
for bad_port in ["22 -oProxyCommand=id", "99999", "abc", "0", "-1"]:
n = node("n1", "10.0.0.1", port=bad_port, protocol=proto)
with pytest.raises(ValueError, match="Invalid TCP SSH port"):
n._get_cmd()
@pytest.mark.parametrize("proto", ["ssh", "sftp"])
def test_dangerous_ssh_directives_in_options_raise_value_error(self, proto):
"""Directives like ProxyCommand, LocalCommand, etc. in options raise ValueError."""
dangerous_opts = [
'-o ProxyCommand="sh -c id"',
"-oProxyCommand=id",
'-o LocalCommand="id"',
"-o PermitLocalCommand=yes",
'-o KnownHostsCommand="id"',
'-o "ProxyCommand=id"',
"-o proxycommand=id",
]
for opt in dangerous_opts:
n = node("n1", "10.0.0.1", options=opt, protocol=proto)
with pytest.raises(ValueError, match="Disallowed SSH directive in options"):
n._get_cmd()
@pytest.mark.parametrize("proto", ["ssh", "sftp", "telnet", "kubectl", "docker", "ssm"])
def test_unbalanced_quotes_in_options_raise_value_error(self, proto):
"""Unbalanced quotes in options raise ValueError."""
n = node("n1", "10.0.0.1", options='-o "StrictHostKeyChecking=no', protocol=proto)
with pytest.raises(ValueError, match="unbalanced quotes"):
n._get_cmd()
def test_ssm_invalid_region_or_profile_tags_raise_value_error(self):
"""SSM tags 'region' and 'profile' reject leading dash and shell metacharacters."""
n_reg = node("ssm1", "i-1234567890abcdef0", protocol="ssm", tags={"region": "us-east-1; id"})
with pytest.raises(ValueError, match="Invalid characters in SSM tag"):
n_reg._get_cmd()
n_prof = node("ssm2", "i-1234567890abcdef0", protocol="ssm", tags={"profile": "-bad"})
with pytest.raises(ValueError, match="cannot start with '-'"):
n_prof._get_cmd()
def test_container_command_validation(self):
"""Kube and Docker commands reject newlines and unbalanced quotes."""
n_kube = node("k1", "pod-1", protocol="kubectl", tags={"kube_command": 'sh -c "echo\nid"'})
with pytest.raises(ValueError, match="cannot contain newlines or null bytes"):
n_kube._get_cmd()
n_kube_q = node("k2", "pod-1", protocol="kubectl", tags={"kube_command": 'sh -c "unclosed'})
with pytest.raises(ValueError, match="unbalanced quotes"):
n_kube_q._get_cmd()
n_doc = node("d1", "cont-1", protocol="docker", tags={"docker_command": 'sh -c "echo\nid"'})
with pytest.raises(ValueError, match="cannot contain newlines or null bytes"):
n_doc._get_cmd()
n_doc_q = node("d2", "cont-1", protocol="docker", tags={"docker_command": 'sh -c "unclosed'})
with pytest.raises(ValueError, match="unbalanced quotes"):
n_doc_q._get_cmd()
# =========================================================================
# 3. Tests Positivos de Regresión y allow_unsafe_options=True
# =========================================================================
class TestPositiveAndAllowUnsafeOptions:
def test_ssh_jumphost_proxycommand_allowed_when_safe_mode(self, populated_config):
"""Valid jumphost chain still generates -o ProxyCommand correctly when allow_unsafe_options=False."""
# populated_config has 'router1'
n = node("target", "10.0.0.2", protocol="ssh", jumphost="router1", config=populated_config)
cmd = n._get_cmd()
assert '-o ProxyCommand="' in cmd
assert "ssh -W %h:%p" in cmd
assert "admin@10.0.0.1" in cmd
def test_ssh_safe_options_permitted(self):
"""Benign SSH options work without issue."""
safe_opts = "-o StrictHostKeyChecking=no -i ~/.ssh/id_rsa -L 8080:localhost:80"
n = node("s1", "10.0.0.1", protocol="ssh", options=safe_opts)
cmd = n._get_cmd()
assert safe_opts in cmd
def test_user_domain_and_ipv6_bracketed_permitted(self):
"""Formats like DOMAIN\\user and bracketed IPv6 [2001:db8::1] remain supported."""
n = node("s1", "[2001:db8::1]", user=r"DOMAIN\user", protocol="ssh", port="22")
cmd = n._get_cmd()
assert r"DOMAIN\user@[2001:db8::1]" in cmd
assert "-p 22" in cmd
def test_allow_unsafe_options_via_config(self, populated_config):
"""When config.config['allow_unsafe_options'] is True, ProxyCommand in options is allowed."""
populated_config.config["allow_unsafe_options"] = True
custom_proxy = '-o ProxyCommand="ssh -W %h:%p bastion"'
n = node("s1", "10.0.0.1", protocol="ssh", options=custom_proxy, config=populated_config)
cmd = n._get_cmd()
assert custom_proxy in cmd
def test_allow_unsafe_options_via_kwarg(self):
"""When allow_unsafe_options=True passed to node, dangerous options are allowed."""
custom_proxy = '-o ProxyCommand="ssh -W %h:%p bastion"'
n = node("s1", "10.0.0.1", protocol="ssh", options=custom_proxy, allow_unsafe_options=True)
cmd = n._get_cmd()
assert custom_proxy in cmd
+27 -11
View File
@@ -150,7 +150,7 @@ el.replaceWith(d);
import yaml
try:
# 1. Parse YAML
data = yaml.load(playbook_yaml, Loader=yaml.FullLoader)
data = yaml.safe_load(playbook_yaml)
except Exception as e:
return {&#34;valid&#34;: False, &#34;error&#34;: f&#34;YAML Syntax Error: {e}&#34;}
@@ -473,7 +473,7 @@ el.replaceWith(d);
import yaml
try:
# 1. Parse YAML
data = yaml.load(playbook_yaml, Loader=yaml.FullLoader)
data = yaml.safe_load(playbook_yaml)
except Exception as e:
return {&#34;valid&#34;: False, &#34;error&#34;: f&#34;YAML Syntax Error: {e}&#34;}
@@ -638,6 +638,9 @@ class ai:
self.session_path = os.path.join(self.sessions_dir, f&#34;{self.session_id}.json&#34;) if self.session_id else None
# Agnostic base prompts
self._update_base_prompts()
def _update_base_prompts(self):
architect_instructions = &#34;&#34;
if self.has_architect:
architect_instructions = &#34;&#34;&#34;
@@ -678,7 +681,7 @@ class ai:
- NO WANDERING: Do not speculate. If stuck, report attempts.
- SAFETY: When you use &#39;run_commands&#39; with configuration commands, the system automatically prompts the user for confirmation. Just execute - don&#39;t ask permission first.
{architect_instructions}
Network Context: {{self.long_term_memory if self.long_term_memory else &#34;Empty.&#34;}}
Network Context: {self.long_term_memory if self.long_term_memory else &#34;Empty.&#34;}
&#34;&#34;&#34;).strip()
self._architect_base_prompt = dedent(f&#34;&#34;&#34;
@@ -718,6 +721,7 @@ class ai:
@property
def engineer_system_prompt(self):
&#34;&#34;&#34;Build engineer system prompt with plugin extensions.&#34;&#34;&#34;
self._update_base_prompts()
if self.engineer_prompt_extensions:
extensions = &#34;\n&#34;.join(self.engineer_prompt_extensions)
return self._engineer_base_prompt + f&#34;\n\nPlugin Capabilities:\n{extensions}&#34;
@@ -726,6 +730,7 @@ class ai:
@property
def architect_system_prompt(self):
&#34;&#34;&#34;Build architect system prompt with plugin extensions.&#34;&#34;&#34;
self._update_base_prompts()
prompt = self._architect_base_prompt
if getattr(self, &#34;one_shot&#34;, False):
prompt += &#34;\n\nCRITICAL 1-SHOT DIAGNOSTICS DIRECTIVE:\nYou are running in a 1-shot offline diagnostics mode. There is no active conversation loop, and you are NOT conversing with a Network Engineer. You MUST deliver your complete strategic analysis immediately and directly to the user. Do not suggest or attempt to delegate/return control to the engineer.&#34;
@@ -1065,6 +1070,7 @@ class ai:
# Reload memory after update
with open(self.memory_path, &#34;r&#34;) as f:
self.long_term_memory = f.read()
self._update_base_prompts()
return &#34;Memory updated successfully.&#34;
except PermissionError as e:
@@ -1300,7 +1306,7 @@ class ai:
# Silently fail for LLM tools
pass
if self.architect_key:
if self.has_architect:
base_tools.extend([
{&#34;type&#34;: &#34;function&#34;, &#34;function&#34;: {&#34;name&#34;: &#34;consult_architect&#34;, &#34;description&#34;: &#34;Ask the Strategic Reasoning Engine for advice on complex design, architecture, or troubleshooting decisions. You remain in control and will present the response to the user. Use this for: configuration planning, design validation, complex troubleshooting.&#34;, &#34;parameters&#34;: {&#34;type&#34;: &#34;object&#34;, &#34;properties&#34;: {&#34;question&#34;: {&#34;type&#34;: &#34;string&#34;, &#34;description&#34;: &#34;Strategic question or decision needed.&#34;}, &#34;technical_summary&#34;: {&#34;type&#34;: &#34;string&#34;, &#34;description&#34;: &#34;Technical findings and context gathered so far.&#34;}}, &#34;required&#34;: [&#34;question&#34;, &#34;technical_summary&#34;]}}},
{&#34;type&#34;: &#34;function&#34;, &#34;function&#34;: {&#34;name&#34;: &#34;escalate_to_architect&#34;, &#34;description&#34;: &#34;Transfer full control to the Strategic Reasoning Engine. Use ONLY when the user explicitly requests the Architect or when the problem requires strategic oversight beyond consultation. After escalation, the Architect takes over the conversation.&#34;, &#34;parameters&#34;: {&#34;type&#34;: &#34;object&#34;, &#34;properties&#34;: {&#34;reason&#34;: {&#34;type&#34;: &#34;string&#34;, &#34;description&#34;: &#34;Why you&#39;re escalating (e.g. &#39;User requested Architect&#39;, &#39;Complex multi-site design needed&#39;).&#34;}, &#34;context&#34;: {&#34;type&#34;: &#34;string&#34;, &#34;description&#34;: &#34;Full context and findings to hand over.&#34;}}, &#34;required&#34;: [&#34;reason&#34;, &#34;context&#34;]}}}
@@ -1657,14 +1663,17 @@ class ai:
if status: update_status(&#34;[architect]Engineer consulting Architect...&#34;)
try:
# Consultation only - Engineer stays in control
auth_kwargs = dict(self.architect_auth or {})
if self.architect_key and &#34;api_key&#34; not in auth_kwargs:
auth_kwargs[&#34;api_key&#34;] = self.architect_key
claude_resp = completion(
model=self.architect_model,
messages=[
{&#34;role&#34;: &#34;system&#34;, &#34;content&#34;: self.architect_system_prompt},
{&#34;role&#34;: &#34;user&#34;, &#34;content&#34;: f&#34;The Engineer needs your strategic advice.\n\nTECHNICAL SUMMARY: {args[&#39;technical_summary&#39;]}\n\nQUESTION: {args[&#39;question&#39;]}\n\nProvide strategic guidance. The Engineer will continue handling the user.&#34;}
],
api_key=self.architect_key,
num_retries=3
num_retries=3,
**auth_kwargs
)
obs = claude_resp.choices[0].message.content
if debug:
@@ -2101,11 +2110,11 @@ Node: {node_name}&#34;&#34;&#34;
<dl>
<dt id="connpy.ai.ai.SAFE_COMMANDS"><code class="name">var <span class="ident">SAFE_COMMANDS</span></code></dt>
<dd>
<div class="desc"><p>The type of the None singleton.</p></div>
<div class="desc"></div>
</dd>
<dt id="connpy.ai.ai.deferred_class_hooks"><code class="name">var <span class="ident">deferred_class_hooks</span></code></dt>
<dd>
<div class="desc"><p>The type of the None singleton.</p></div>
<div class="desc"></div>
</dd>
</dl>
<h3>Instance variables</h3>
@@ -2119,6 +2128,7 @@ Node: {node_name}&#34;&#34;&#34;
<pre><code class="python">@property
def architect_system_prompt(self):
&#34;&#34;&#34;Build architect system prompt with plugin extensions.&#34;&#34;&#34;
self._update_base_prompts()
prompt = self._architect_base_prompt
if getattr(self, &#34;one_shot&#34;, False):
prompt += &#34;\n\nCRITICAL 1-SHOT DIAGNOSTICS DIRECTIVE:\nYou are running in a 1-shot offline diagnostics mode. There is no active conversation loop, and you are NOT conversing with a Network Engineer. You MUST deliver your complete strategic analysis immediately and directly to the user. Do not suggest or attempt to delegate/return control to the engineer.&#34;
@@ -2138,6 +2148,7 @@ def architect_system_prompt(self):
<pre><code class="python">@property
def engineer_system_prompt(self):
&#34;&#34;&#34;Build engineer system prompt with plugin extensions.&#34;&#34;&#34;
self._update_base_prompts()
if self.engineer_prompt_extensions:
extensions = &#34;\n&#34;.join(self.engineer_prompt_extensions)
return self._engineer_base_prompt + f&#34;\n\nPlugin Capabilities:\n{extensions}&#34;
@@ -2661,14 +2672,17 @@ def ask(self, user_input, dryrun=False, chat_history=None, status=None, debug=Fa
if status: update_status(&#34;[architect]Engineer consulting Architect...&#34;)
try:
# Consultation only - Engineer stays in control
auth_kwargs = dict(self.architect_auth or {})
if self.architect_key and &#34;api_key&#34; not in auth_kwargs:
auth_kwargs[&#34;api_key&#34;] = self.architect_key
claude_resp = completion(
model=self.architect_model,
messages=[
{&#34;role&#34;: &#34;system&#34;, &#34;content&#34;: self.architect_system_prompt},
{&#34;role&#34;: &#34;user&#34;, &#34;content&#34;: f&#34;The Engineer needs your strategic advice.\n\nTECHNICAL SUMMARY: {args[&#39;technical_summary&#39;]}\n\nQUESTION: {args[&#39;question&#39;]}\n\nProvide strategic guidance. The Engineer will continue handling the user.&#34;}
],
api_key=self.architect_key,
num_retries=3
num_retries=3,
**auth_kwargs
)
obs = claude_resp.choices[0].message.content
if debug:
@@ -2976,6 +2990,7 @@ def confirm(self, user_input): return True</code></pre>
# Reload memory after update
with open(self.memory_path, &#34;r&#34;) as f:
self.long_term_memory = f.read()
self._update_base_prompts()
return &#34;Memory updated successfully.&#34;
except PermissionError as e:
@@ -3024,8 +3039,9 @@ def confirm(self, user_input): return True</code></pre>
self.tool_status_formatters[name] = status_formatter</code></pre>
</details>
<div class="desc"><p>Register an external tool for the AI system.</p>
<h2 id="args">Args</h2>
<dl>
<dt>Args</dt>
<dt>-----=</dt>
<dt><strong><code>tool_definition</code></strong> :&ensp;<code>dict</code></dt>
<dd>OpenAI-compatible tool definition.</dd>
<dt><strong><code>handler</code></strong> :&ensp;<code>callable</code></dt>
File diff suppressed because it is too large. Load diff
+99 -16
View File
@@ -83,7 +83,7 @@ el.replaceWith(d);
try:
self.app.services.ai.delete_session(args.delete_session[0])
printer.success(f&#34;Session {args.delete_session[0]} deleted.&#34;)
except Exception as e:
except Exception as e: # noqa: BLE001
printer.error(str(e))
return
@@ -236,7 +236,7 @@ el.replaceWith(d);
try:
self.app.services.ai.configure_mcp(name, url=url, auto_load_on_os=os_filter)
printer.success(f&#34;MCP server &#39;{name}&#39; added/updated.&#34;)
except Exception as e:
except Exception as e: # noqa: BLE001
printer.error(str(e))
return
@@ -248,7 +248,7 @@ el.replaceWith(d);
try:
self.app.services.ai.configure_mcp(name, remove=True)
printer.success(f&#34;MCP server &#39;{name}&#39; removed.&#34;)
except Exception as e:
except Exception as e: # noqa: BLE001
printer.error(str(e))
return
@@ -261,13 +261,54 @@ el.replaceWith(d);
try:
self.app.services.ai.configure_mcp(name, enabled=enabled)
printer.success(f&#34;MCP server &#39;{name}&#39; {&#39;enabled&#39; if enabled else &#39;disabled&#39;}.&#34;)
except Exception as e:
except Exception as e: # noqa: BLE001
printer.error(str(e))
return
elif action == &#34;serve&#34;:
transport = &#34;stdio&#34;
host = &#34;127.0.0.1&#34;
port = 8000
trust = getattr(args, &#34;trust&#34;, False)
idx = 1
while idx &lt; len(mcp_args):
tok = mcp_args[idx]
if tok in [&#34;--sse&#34;, &#34;sse&#34;]:
transport = &#34;sse&#34;
elif tok in [&#34;--stdio&#34;, &#34;stdio&#34;]:
transport = &#34;stdio&#34;
elif tok in [&#34;--trust&#34;, &#34;trust&#34;]:
trust = True
elif tok in [&#34;--port&#34;, &#34;-p&#34;]:
idx += 1
if idx &lt; len(mcp_args):
try:
port = int(mcp_args[idx])
except ValueError:
printer.error(f&#34;Invalid port: {mcp_args[idx]}&#34;)
return
elif tok.startswith(&#34;--port=&#34;):
try:
port = int(tok.split(&#34;=&#34;, 1)[1])
except ValueError:
printer.error(f&#34;Invalid port: {tok}&#34;)
return
elif tok in [&#34;--host&#34;, &#34;-h&#34;]:
idx += 1
if idx &lt; len(mcp_args):
host = mcp_args[idx]
elif tok.startswith(&#34;--host=&#34;):
host = tok.split(&#34;=&#34;, 1)[1]
idx += 1
from ..mcp_server import ConnpyMCPServer
server = ConnpyMCPServer(config=self.app.config, services=self.app.services, trust=trust)
return server.run(transport=transport, host=host, port=port)
else:
printer.error(f&#34;Unknown MCP action: {action}&#34;)
printer.info(&#34;Available actions: list, add, remove, enable, disable&#34;)
printer.info(&#34;Available actions: list, add, remove, enable, disable, serve&#34;)
return
# 2. Interactive Wizard Mode (if no arguments provided)
@@ -309,15 +350,16 @@ el.replaceWith(d);
self.app.services.ai.configure_mcp(result[&#34;name&#34;], remove=True)
printer.success(f&#34;MCP server &#39;{result[&#39;name&#39;]}&#39; removed.&#34;)
except Exception as e:
except Exception as e: # noqa: BLE001
printer.error(str(e))
def _parse_auth_value(self, value):
if not value or value.lower() in [&#34;none&#34;, &#34;clear&#34;]:
return None
import os
import yaml
import json
import os
import yaml
if os.path.exists(value):
try:
with open(value, &#34;r&#34;) as f:
@@ -326,7 +368,7 @@ el.replaceWith(d);
return json.loads(content)
except ValueError:
return yaml.safe_load(content)
except Exception as e:
except Exception as e: # noqa: BLE001
printer.error(f&#34;Failed to read/parse auth file &#39;{value}&#39;: {e}&#34;)
sys.exit(1)
@@ -338,7 +380,7 @@ el.replaceWith(d);
if isinstance(parsed, dict):
return parsed
raise ValueError()
except Exception:
except Exception: # noqa: BLE001
printer.error(&#34;Auth parameter must be a valid JSON/YAML string, or a path to a JSON/YAML file.&#34;)
sys.exit(1)</code></pre>
</details>
@@ -387,7 +429,7 @@ el.replaceWith(d);
try:
self.app.services.ai.configure_mcp(name, url=url, auto_load_on_os=os_filter)
printer.success(f&#34;MCP server &#39;{name}&#39; added/updated.&#34;)
except Exception as e:
except Exception as e: # noqa: BLE001
printer.error(str(e))
return
@@ -399,7 +441,7 @@ el.replaceWith(d);
try:
self.app.services.ai.configure_mcp(name, remove=True)
printer.success(f&#34;MCP server &#39;{name}&#39; removed.&#34;)
except Exception as e:
except Exception as e: # noqa: BLE001
printer.error(str(e))
return
@@ -412,13 +454,54 @@ el.replaceWith(d);
try:
self.app.services.ai.configure_mcp(name, enabled=enabled)
printer.success(f&#34;MCP server &#39;{name}&#39; {&#39;enabled&#39; if enabled else &#39;disabled&#39;}.&#34;)
except Exception as e:
except Exception as e: # noqa: BLE001
printer.error(str(e))
return
elif action == &#34;serve&#34;:
transport = &#34;stdio&#34;
host = &#34;127.0.0.1&#34;
port = 8000
trust = getattr(args, &#34;trust&#34;, False)
idx = 1
while idx &lt; len(mcp_args):
tok = mcp_args[idx]
if tok in [&#34;--sse&#34;, &#34;sse&#34;]:
transport = &#34;sse&#34;
elif tok in [&#34;--stdio&#34;, &#34;stdio&#34;]:
transport = &#34;stdio&#34;
elif tok in [&#34;--trust&#34;, &#34;trust&#34;]:
trust = True
elif tok in [&#34;--port&#34;, &#34;-p&#34;]:
idx += 1
if idx &lt; len(mcp_args):
try:
port = int(mcp_args[idx])
except ValueError:
printer.error(f&#34;Invalid port: {mcp_args[idx]}&#34;)
return
elif tok.startswith(&#34;--port=&#34;):
try:
port = int(tok.split(&#34;=&#34;, 1)[1])
except ValueError:
printer.error(f&#34;Invalid port: {tok}&#34;)
return
elif tok in [&#34;--host&#34;, &#34;-h&#34;]:
idx += 1
if idx &lt; len(mcp_args):
host = mcp_args[idx]
elif tok.startswith(&#34;--host=&#34;):
host = tok.split(&#34;=&#34;, 1)[1]
idx += 1
from ..mcp_server import ConnpyMCPServer
server = ConnpyMCPServer(config=self.app.config, services=self.app.services, trust=trust)
return server.run(transport=transport, host=host, port=port)
else:
printer.error(f&#34;Unknown MCP action: {action}&#34;)
printer.info(&#34;Available actions: list, add, remove, enable, disable&#34;)
printer.info(&#34;Available actions: list, add, remove, enable, disable, serve&#34;)
return
# 2. Interactive Wizard Mode (if no arguments provided)
@@ -460,7 +543,7 @@ el.replaceWith(d);
self.app.services.ai.configure_mcp(result[&#34;name&#34;], remove=True)
printer.success(f&#34;MCP server &#39;{result[&#39;name&#39;]}&#39; removed.&#34;)
except Exception as e:
except Exception as e: # noqa: BLE001
printer.error(str(e))</code></pre>
</details>
<div class="desc"><p>Handle MCP server configuration via CLI tokens or interactive wizard.</p></div>
@@ -497,7 +580,7 @@ el.replaceWith(d);
try:
self.app.services.ai.delete_session(args.delete_session[0])
printer.success(f&#34;Session {args.delete_session[0]} deleted.&#34;)
except Exception as e:
except Exception as e: # noqa: BLE001
printer.error(str(e))
return
+164 -4
View File
@@ -80,7 +80,12 @@ el.replaceWith(d);
&#34;sync_remote&#34;: self.set_sync_remote,
&#34;shell_command&#34;: self.set_shell_config,
&#34;shell_prompt&#34;: self.set_shell_config,
&#34;shell_os&#34;: self.set_shell_config
&#34;shell_os&#34;: self.set_shell_config,
&#34;max_auth_prompts&#34;: self.set_auth_budget,
&#34;max_total_auth_attempts&#34;: self.set_auth_budget,
&#34;max_hop_host_attempts&#34;: self.set_auth_budget,
&#34;max_global_user_attempts&#34;: self.set_auth_budget,
&#34;disable_fallback_on_insecure_hostkey&#34;: self.set_disable_fallback_on_insecure_hostkey,
}
handler = actions.get(getattr(args, &#34;command&#34;, None))
if handler:
@@ -251,7 +256,71 @@ el.replaceWith(d);
self.app.services.config_svc.update_setting(&#34;shell&#34;, shell_cfg)
printer.success(&#34;Config saved&#34;)
except (ConnpyError, InvalidConfigurationError) as e:
printer.error(str(e))</code></pre>
printer.error(str(e))
def set_auth_budget(self, args):
key = args.command
raw_val = args.data[0] if isinstance(args.data, list) else args.data
bounds_map = {
&#34;max_auth_prompts&#34;: (MIN_AUTH_PROMPTS, MAX_AUTH_PROMPTS_CAP),
&#34;max_total_auth_attempts&#34;: (MIN_TOTAL_AUTH_ATTEMPTS, MAX_TOTAL_AUTH_ATTEMPTS_CAP),
&#34;max_hop_host_attempts&#34;: (MIN_HOP_HOST_ATTEMPTS, MAX_HOP_HOST_ATTEMPTS_CAP),
&#34;max_global_user_attempts&#34;: (1, 1000),
}
if key not in bounds_map:
raise InvalidConfigurationError(f&#34;Unknown auth budget setting: {key}&#34;)
if isinstance(raw_val, bool):
msg = f&#34;Invalid {key}: boolean is not permitted&#34;
printer.error(msg)
raise InvalidConfigurationError(msg)
if isinstance(raw_val, int):
val = raw_val
elif isinstance(raw_val, str) and re.fullmatch(r&#34;[+-]?\d+&#34;, raw_val.strip()):
val = int(raw_val.strip())
else:
msg = f&#34;Invalid {key}: expected integer, got {raw_val!r}&#34;
printer.error(msg)
raise InvalidConfigurationError(msg)
if val == 0:
cfg_obj = self.app.config
if not hasattr(cfg_obj, &#34;assert_called&#34;) and callable(
getattr(cfg_obj, &#34;transactional_update&#34;, None)
):
cfg_obj.transactional_update(lambda c: c.config.pop(key, None))
else:
cfg_obj.config.pop(key, None)
cfg_obj._saveconfig(cfg_obj.file)
printer.success(&#34;Config saved&#34;)
return
min_v, max_v = bounds_map[key]
try:
validate_strict_int_range(val, key, min_v, max_v)
except ValueError as exc:
printer.error(str(exc))
raise InvalidConfigurationError(str(exc)) from None
self.app.services.config_svc.update_setting(key, val)
printer.success(&#34;Config saved&#34;)
def set_disable_fallback_on_insecure_hostkey(self, args):
raw_val = args.data[0] if isinstance(args.data, list) else args.data
if isinstance(raw_val, bool):
val = raw_val
elif isinstance(raw_val, str) and raw_val.strip().lower() in (&#34;true&#34;, &#34;false&#34;):
val = raw_val.strip().lower() == &#34;true&#34;
else:
msg = &#34;Invalid disable_fallback_on_insecure_hostkey: expected &#39;true&#39; or &#39;false&#39;&#34;
printer.error(msg)
raise InvalidConfigurationError(msg)
self.app.services.config_svc.update_setting(
&#34;disable_fallback_on_insecure_hostkey&#34;,
val,
)
printer.success(&#34;Config saved&#34;)</code></pre>
</details>
<div class="desc"></div>
<h3>Methods</h3>
@@ -285,7 +354,12 @@ el.replaceWith(d);
&#34;sync_remote&#34;: self.set_sync_remote,
&#34;shell_command&#34;: self.set_shell_config,
&#34;shell_prompt&#34;: self.set_shell_config,
&#34;shell_os&#34;: self.set_shell_config
&#34;shell_os&#34;: self.set_shell_config,
&#34;max_auth_prompts&#34;: self.set_auth_budget,
&#34;max_total_auth_attempts&#34;: self.set_auth_budget,
&#34;max_hop_host_attempts&#34;: self.set_auth_budget,
&#34;max_global_user_attempts&#34;: self.set_auth_budget,
&#34;disable_fallback_on_insecure_hostkey&#34;: self.set_disable_fallback_on_insecure_hostkey,
}
handler = actions.get(getattr(args, &#34;command&#34;, None))
if handler:
@@ -333,6 +407,64 @@ el.replaceWith(d);
</details>
<div class="desc"></div>
</dd>
<dt id="connpy.cli.config_handler.ConfigHandler.set_auth_budget"><code class="name flex">
<span>def <span class="ident">set_auth_budget</span></span>(<span>self, args)</span>
</code></dt>
<dd>
<details class="source">
<summary>
<span>Expand source code</span>
</summary>
<pre><code class="python">def set_auth_budget(self, args):
key = args.command
raw_val = args.data[0] if isinstance(args.data, list) else args.data
bounds_map = {
&#34;max_auth_prompts&#34;: (MIN_AUTH_PROMPTS, MAX_AUTH_PROMPTS_CAP),
&#34;max_total_auth_attempts&#34;: (MIN_TOTAL_AUTH_ATTEMPTS, MAX_TOTAL_AUTH_ATTEMPTS_CAP),
&#34;max_hop_host_attempts&#34;: (MIN_HOP_HOST_ATTEMPTS, MAX_HOP_HOST_ATTEMPTS_CAP),
&#34;max_global_user_attempts&#34;: (1, 1000),
}
if key not in bounds_map:
raise InvalidConfigurationError(f&#34;Unknown auth budget setting: {key}&#34;)
if isinstance(raw_val, bool):
msg = f&#34;Invalid {key}: boolean is not permitted&#34;
printer.error(msg)
raise InvalidConfigurationError(msg)
if isinstance(raw_val, int):
val = raw_val
elif isinstance(raw_val, str) and re.fullmatch(r&#34;[+-]?\d+&#34;, raw_val.strip()):
val = int(raw_val.strip())
else:
msg = f&#34;Invalid {key}: expected integer, got {raw_val!r}&#34;
printer.error(msg)
raise InvalidConfigurationError(msg)
if val == 0:
cfg_obj = self.app.config
if not hasattr(cfg_obj, &#34;assert_called&#34;) and callable(
getattr(cfg_obj, &#34;transactional_update&#34;, None)
):
cfg_obj.transactional_update(lambda c: c.config.pop(key, None))
else:
cfg_obj.config.pop(key, None)
cfg_obj._saveconfig(cfg_obj.file)
printer.success(&#34;Config saved&#34;)
return
min_v, max_v = bounds_map[key]
try:
validate_strict_int_range(val, key, min_v, max_v)
except ValueError as exc:
printer.error(str(exc))
raise InvalidConfigurationError(str(exc)) from None
self.app.services.config_svc.update_setting(key, val)
printer.success(&#34;Config saved&#34;)</code></pre>
</details>
<div class="desc"></div>
</dd>
<dt id="connpy.cli.config_handler.ConfigHandler.set_case"><code class="name flex">
<span>def <span class="ident">set_case</span></span>(<span>self, args)</span>
</code></dt>
@@ -367,6 +499,32 @@ el.replaceWith(d);
</details>
<div class="desc"></div>
</dd>
<dt id="connpy.cli.config_handler.ConfigHandler.set_disable_fallback_on_insecure_hostkey"><code class="name flex">
<span>def <span class="ident">set_disable_fallback_on_insecure_hostkey</span></span>(<span>self, args)</span>
</code></dt>
<dd>
<details class="source">
<summary>
<span>Expand source code</span>
</summary>
<pre><code class="python">def set_disable_fallback_on_insecure_hostkey(self, args):
raw_val = args.data[0] if isinstance(args.data, list) else args.data
if isinstance(raw_val, bool):
val = raw_val
elif isinstance(raw_val, str) and raw_val.strip().lower() in (&#34;true&#34;, &#34;false&#34;):
val = raw_val.strip().lower() == &#34;true&#34;
else:
msg = &#34;Invalid disable_fallback_on_insecure_hostkey: expected &#39;true&#39; or &#39;false&#39;&#34;
printer.error(msg)
raise InvalidConfigurationError(msg)
self.app.services.config_svc.update_setting(
&#34;disable_fallback_on_insecure_hostkey&#34;,
val,
)
printer.success(&#34;Config saved&#34;)</code></pre>
</details>
<div class="desc"></div>
</dd>
<dt id="connpy.cli.config_handler.ConfigHandler.set_fzf"><code class="name flex">
<span>def <span class="ident">set_fzf</span></span>(<span>self, args)</span>
</code></dt>
@@ -577,11 +735,13 @@ el.replaceWith(d);
<ul>
<li>
<h4><code><a title="connpy.cli.config_handler.ConfigHandler" href="#connpy.cli.config_handler.ConfigHandler">ConfigHandler</a></code></h4>
<ul class="two-column">
<ul class="">
<li><code><a title="connpy.cli.config_handler.ConfigHandler.dispatch" href="#connpy.cli.config_handler.ConfigHandler.dispatch">dispatch</a></code></li>
<li><code><a title="connpy.cli.config_handler.ConfigHandler.set_ai_config" href="#connpy.cli.config_handler.ConfigHandler.set_ai_config">set_ai_config</a></code></li>
<li><code><a title="connpy.cli.config_handler.ConfigHandler.set_auth_budget" href="#connpy.cli.config_handler.ConfigHandler.set_auth_budget">set_auth_budget</a></code></li>
<li><code><a title="connpy.cli.config_handler.ConfigHandler.set_case" href="#connpy.cli.config_handler.ConfigHandler.set_case">set_case</a></code></li>
<li><code><a title="connpy.cli.config_handler.ConfigHandler.set_configfolder" href="#connpy.cli.config_handler.ConfigHandler.set_configfolder">set_configfolder</a></code></li>
<li><code><a title="connpy.cli.config_handler.ConfigHandler.set_disable_fallback_on_insecure_hostkey" href="#connpy.cli.config_handler.ConfigHandler.set_disable_fallback_on_insecure_hostkey">set_disable_fallback_on_insecure_hostkey</a></code></li>
<li><code><a title="connpy.cli.config_handler.ConfigHandler.set_fzf" href="#connpy.cli.config_handler.ConfigHandler.set_fzf">set_fzf</a></code></li>
<li><code><a title="connpy.cli.config_handler.ConfigHandler.set_idletime" href="#connpy.cli.config_handler.ConfigHandler.set_idletime">set_idletime</a></code></li>
<li><code><a title="connpy.cli.config_handler.ConfigHandler.set_remote_host" href="#connpy.cli.config_handler.ConfigHandler.set_remote_host">set_remote_host</a></code></li>
+122 -30
View File
@@ -72,6 +72,7 @@ el.replaceWith(d);
questions.append(inquirer.Confirm(&#34;jumphost&#34;, message=&#34;Edit jumphost?&#34;))
questions.append(inquirer.Confirm(&#34;user&#34;, message=&#34;Edit User?&#34;))
questions.append(inquirer.Confirm(&#34;password&#34;, message=&#34;Edit password?&#34;))
questions.append(inquirer.Confirm(&#34;fallback&#34;, message=&#34;Edit fallback credentials?&#34;))
return inquirer.prompt(questions)
def questions_nodes(self, unique, uniques=None, edit=None):
@@ -82,11 +83,13 @@ el.replaceWith(d);
defaults[&#34;tags&#34;] = &#34;&#34;
if &#34;jumphost&#34; not in defaults:
defaults[&#34;jumphost&#34;] = &#34;&#34;
except Exception:
defaults = {&#34;host&#34;: &#34;&#34;, &#34;protocol&#34;: &#34;&#34;, &#34;port&#34;: &#34;&#34;, &#34;user&#34;: &#34;&#34;, &#34;options&#34;: &#34;&#34;, &#34;logs&#34;: &#34;&#34;, &#34;tags&#34;: &#34;&#34;, &#34;password&#34;: &#34;&#34;, &#34;jumphost&#34;: &#34;&#34;}
if &#34;fallback&#34; not in defaults:
defaults[&#34;fallback&#34;] = &#34;&#34;
except Exception: # noqa: BLE001
defaults = {&#34;host&#34;: &#34;&#34;, &#34;protocol&#34;: &#34;&#34;, &#34;port&#34;: &#34;&#34;, &#34;user&#34;: &#34;&#34;, &#34;options&#34;: &#34;&#34;, &#34;logs&#34;: &#34;&#34;, &#34;tags&#34;: &#34;&#34;, &#34;password&#34;: &#34;&#34;, &#34;jumphost&#34;: &#34;&#34;, &#34;fallback&#34;: &#34;&#34;}
node = {}
if edit is None:
edit = {&#34;host&#34;: True, &#34;protocol&#34;: True, &#34;port&#34;: True, &#34;user&#34;: True, &#34;password&#34;: True, &#34;options&#34;: True, &#34;logs&#34;: True, &#34;tags&#34;: True, &#34;jumphost&#34;: True}
edit = {&#34;host&#34;: True, &#34;protocol&#34;: True, &#34;port&#34;: True, &#34;user&#34;: True, &#34;password&#34;: True, &#34;options&#34;: True, &#34;logs&#34;: True, &#34;tags&#34;: True, &#34;jumphost&#34;: True, &#34;fallback&#34;: True}
questions = []
if edit[&#34;host&#34;]:
questions.append(inquirer.Text(&#34;host&#34;, message=&#34;Add Hostname or IP&#34;, validate=self.validators.host_validation, default=defaults[&#34;host&#34;]))
@@ -124,6 +127,8 @@ el.replaceWith(d);
questions.append(inquirer.List(&#34;password&#34;, message=&#34;Password: Use a local password, no password or a list of profiles to reference?&#34;, choices=[&#34;Local Password&#34;, &#34;Profiles&#34;, &#34;No Password&#34;]))
else:
node[&#34;password&#34;] = defaults[&#34;password&#34;]
if not edit.get(&#34;fallback&#34;):
node[&#34;fallback&#34;] = defaults.get(&#34;fallback&#34;, &#34;&#34;)
answer = inquirer.prompt(questions)
if answer is None:
@@ -141,12 +146,28 @@ el.replaceWith(d);
passa = inquirer.prompt(passq)
if passa is None:
return False
answer[&#34;password&#34;] = passa[&#34;password&#34;].split(&#34;,&#34;)
answer[&#34;password&#34;] = [p.strip() for p in passa[&#34;password&#34;].split(&#34;,&#34;) if p.strip()]
elif answer[&#34;password&#34;] == &#34;No Password&#34;:
answer[&#34;password&#34;] = &#34;&#34;
if &#34;tags&#34; in answer and not answer[&#34;tags&#34;].startswith(&#34;@&#34;) and answer[&#34;tags&#34;]:
answer[&#34;tags&#34;] = ast.literal_eval(answer[&#34;tags&#34;])
if edit.get(&#34;fallback&#34;):
raw_def_fb = defaults.get(&#34;fallback&#34;, &#34;&#34;)
def_fb_str = json.dumps(raw_def_fb) if isinstance(raw_def_fb, (list, dict)) else str(raw_def_fb or &#34;&#34;)
fbq = [
inquirer.Text(
&#34;fallback&#34;,
message=&#34;Add fallback credentials (@profile, @p1,@p2, JSON list/dict, or compose:)&#34;,
validate=self.validators.fallback_validation,
default=def_fb_str.replace(&#34;{&#34;, &#34;{{&#34;).replace(&#34;}&#34;, &#34;}}&#34;),
)
]
fba = inquirer.prompt(fbq)
if fba is None:
return False
answer[&#34;fallback&#34;] = _parse_fallback_input(fba[&#34;fallback&#34;])
if &#34;tags&#34; in answer and answer[&#34;tags&#34;] and not str(answer[&#34;tags&#34;]).startswith(&#34;@&#34;):
answer[&#34;tags&#34;] = _parse_tags_input(answer[&#34;tags&#34;])
result = {**uniques, **answer, **node}
result[&#34;type&#34;] = &#34;connection&#34;
@@ -160,11 +181,13 @@ el.replaceWith(d);
defaults[&#34;tags&#34;] = &#34;&#34;
if &#34;jumphost&#34; not in defaults:
defaults[&#34;jumphost&#34;] = &#34;&#34;
except Exception:
defaults = {&#34;host&#34;: &#34;&#34;, &#34;protocol&#34;: &#34;&#34;, &#34;port&#34;: &#34;&#34;, &#34;user&#34;: &#34;&#34;, &#34;options&#34;: &#34;&#34;, &#34;logs&#34;: &#34;&#34;, &#34;tags&#34;: &#34;&#34;, &#34;jumphost&#34;: &#34;&#34;}
if &#34;fallback&#34; not in defaults:
defaults[&#34;fallback&#34;] = &#34;&#34;
except Exception: # noqa: BLE001
defaults = {&#34;host&#34;: &#34;&#34;, &#34;protocol&#34;: &#34;&#34;, &#34;port&#34;: &#34;&#34;, &#34;user&#34;: &#34;&#34;, &#34;options&#34;: &#34;&#34;, &#34;logs&#34;: &#34;&#34;, &#34;tags&#34;: &#34;&#34;, &#34;jumphost&#34;: &#34;&#34;, &#34;fallback&#34;: &#34;&#34;}
profile = {}
if edit is None:
edit = {&#34;host&#34;: True, &#34;protocol&#34;: True, &#34;port&#34;: True, &#34;user&#34;: True, &#34;password&#34;: True, &#34;options&#34;: True, &#34;logs&#34;: True, &#34;tags&#34;: True, &#34;jumphost&#34;: True}
edit = {&#34;host&#34;: True, &#34;protocol&#34;: True, &#34;port&#34;: True, &#34;user&#34;: True, &#34;password&#34;: True, &#34;options&#34;: True, &#34;logs&#34;: True, &#34;tags&#34;: True, &#34;jumphost&#34;: True, &#34;fallback&#34;: True}
questions = []
if edit[&#34;host&#34;]:
questions.append(inquirer.Text(&#34;host&#34;, message=&#34;Add Hostname or IP&#34;, default=defaults[&#34;host&#34;]))
@@ -202,17 +225,24 @@ el.replaceWith(d);
questions.append(inquirer.Password(&#34;password&#34;, message=&#34;Set Password&#34;))
else:
profile[&#34;password&#34;] = defaults[&#34;password&#34;]
if edit.get(&#34;fallback&#34;):
raw_def_fb = defaults.get(&#34;fallback&#34;, &#34;&#34;)
def_fb_str = json.dumps(raw_def_fb) if isinstance(raw_def_fb, (list, dict)) else str(raw_def_fb or &#34;&#34;)
questions.append(inquirer.Text(&#34;fallback&#34;, message=&#34;Add fallback credentials (@profile, @p1,@p2, JSON list/dict, or compose:)&#34;, validate=self.validators.fallback_validation, default=def_fb_str.replace(&#34;{&#34;, &#34;{{&#34;).replace(&#34;}&#34;, &#34;}}&#34;)))
else:
profile[&#34;fallback&#34;] = defaults.get(&#34;fallback&#34;, &#34;&#34;)
answer = inquirer.prompt(questions)
if answer is None:
return False
if &#34;password&#34; in answer:
if answer[&#34;password&#34;] != &#34;&#34;:
if &#34;password&#34; in answer and answer[&#34;password&#34;] != &#34;&#34;:
answer[&#34;password&#34;] = self.app.services.config_svc.encrypt_password(answer[&#34;password&#34;])
if &#34;tags&#34; in answer and answer[&#34;tags&#34;]:
answer[&#34;tags&#34;] = ast.literal_eval(answer[&#34;tags&#34;])
answer[&#34;tags&#34;] = _parse_tags_input(answer[&#34;tags&#34;])
if &#34;fallback&#34; in answer:
answer[&#34;fallback&#34;] = _parse_fallback_input(answer[&#34;fallback&#34;])
result = {**answer, **profile}
result[&#34;id&#34;] = unique
@@ -241,17 +271,33 @@ el.replaceWith(d);
if answer[&#34;password&#34;] == &#34;Local Password&#34;:
passq = [inquirer.Password(&#34;password&#34;, message=&#34;Set Password&#34;)]
passa = inquirer.prompt(passq)
if passa is None:
return False
answer[&#34;password&#34;] = self.app.services.config_svc.encrypt_password(passa[&#34;password&#34;])
elif answer[&#34;password&#34;] == &#34;Profiles&#34;:
passq = [(inquirer.Text(&#34;password&#34;, message=&#34;Set a @profile or a comma separated list of @profiles&#34;, validate=self.validators.pass_validation))]
passa = inquirer.prompt(passq)
answer[&#34;password&#34;] = passa[&#34;password&#34;].split(&#34;,&#34;)
if passa is None:
return False
answer[&#34;password&#34;] = [p.strip() for p in passa[&#34;password&#34;].split(&#34;,&#34;) if p.strip()]
elif answer[&#34;password&#34;] == &#34;No Password&#34;:
answer[&#34;password&#34;] = &#34;&#34;
fbq = [
inquirer.Text(
&#34;fallback&#34;,
message=&#34;Add fallback credentials (@profile, @p1,@p2, JSON list/dict, or compose:)&#34;,
validate=self.validators.fallback_validation,
)
]
fba = inquirer.prompt(fbq)
if fba is None:
return False
answer[&#34;fallback&#34;] = _parse_fallback_input(fba[&#34;fallback&#34;])
answer[&#34;type&#34;] = &#34;connection&#34;
if &#34;tags&#34; in answer and not answer[&#34;tags&#34;].startswith(&#34;@&#34;) and answer[&#34;tags&#34;]:
answer[&#34;tags&#34;] = ast.literal_eval(answer[&#34;tags&#34;])
if &#34;tags&#34; in answer and answer[&#34;tags&#34;] and not str(answer[&#34;tags&#34;]).startswith(&#34;@&#34;):
answer[&#34;tags&#34;] = _parse_tags_input(answer[&#34;tags&#34;])
return answer
@@ -463,17 +509,33 @@ el.replaceWith(d);
if answer[&#34;password&#34;] == &#34;Local Password&#34;:
passq = [inquirer.Password(&#34;password&#34;, message=&#34;Set Password&#34;)]
passa = inquirer.prompt(passq)
if passa is None:
return False
answer[&#34;password&#34;] = self.app.services.config_svc.encrypt_password(passa[&#34;password&#34;])
elif answer[&#34;password&#34;] == &#34;Profiles&#34;:
passq = [(inquirer.Text(&#34;password&#34;, message=&#34;Set a @profile or a comma separated list of @profiles&#34;, validate=self.validators.pass_validation))]
passa = inquirer.prompt(passq)
answer[&#34;password&#34;] = passa[&#34;password&#34;].split(&#34;,&#34;)
if passa is None:
return False
answer[&#34;password&#34;] = [p.strip() for p in passa[&#34;password&#34;].split(&#34;,&#34;) if p.strip()]
elif answer[&#34;password&#34;] == &#34;No Password&#34;:
answer[&#34;password&#34;] = &#34;&#34;
fbq = [
inquirer.Text(
&#34;fallback&#34;,
message=&#34;Add fallback credentials (@profile, @p1,@p2, JSON list/dict, or compose:)&#34;,
validate=self.validators.fallback_validation,
)
]
fba = inquirer.prompt(fbq)
if fba is None:
return False
answer[&#34;fallback&#34;] = _parse_fallback_input(fba[&#34;fallback&#34;])
answer[&#34;type&#34;] = &#34;connection&#34;
if &#34;tags&#34; in answer and not answer[&#34;tags&#34;].startswith(&#34;@&#34;) and answer[&#34;tags&#34;]:
answer[&#34;tags&#34;] = ast.literal_eval(answer[&#34;tags&#34;])
if &#34;tags&#34; in answer and answer[&#34;tags&#34;] and not str(answer[&#34;tags&#34;]).startswith(&#34;@&#34;):
answer[&#34;tags&#34;] = _parse_tags_input(answer[&#34;tags&#34;])
return answer</code></pre>
</details>
@@ -499,6 +561,7 @@ el.replaceWith(d);
questions.append(inquirer.Confirm(&#34;jumphost&#34;, message=&#34;Edit jumphost?&#34;))
questions.append(inquirer.Confirm(&#34;user&#34;, message=&#34;Edit User?&#34;))
questions.append(inquirer.Confirm(&#34;password&#34;, message=&#34;Edit password?&#34;))
questions.append(inquirer.Confirm(&#34;fallback&#34;, message=&#34;Edit fallback credentials?&#34;))
return inquirer.prompt(questions)</code></pre>
</details>
<div class="desc"></div>
@@ -519,11 +582,13 @@ el.replaceWith(d);
defaults[&#34;tags&#34;] = &#34;&#34;
if &#34;jumphost&#34; not in defaults:
defaults[&#34;jumphost&#34;] = &#34;&#34;
except Exception:
defaults = {&#34;host&#34;: &#34;&#34;, &#34;protocol&#34;: &#34;&#34;, &#34;port&#34;: &#34;&#34;, &#34;user&#34;: &#34;&#34;, &#34;options&#34;: &#34;&#34;, &#34;logs&#34;: &#34;&#34;, &#34;tags&#34;: &#34;&#34;, &#34;password&#34;: &#34;&#34;, &#34;jumphost&#34;: &#34;&#34;}
if &#34;fallback&#34; not in defaults:
defaults[&#34;fallback&#34;] = &#34;&#34;
except Exception: # noqa: BLE001
defaults = {&#34;host&#34;: &#34;&#34;, &#34;protocol&#34;: &#34;&#34;, &#34;port&#34;: &#34;&#34;, &#34;user&#34;: &#34;&#34;, &#34;options&#34;: &#34;&#34;, &#34;logs&#34;: &#34;&#34;, &#34;tags&#34;: &#34;&#34;, &#34;password&#34;: &#34;&#34;, &#34;jumphost&#34;: &#34;&#34;, &#34;fallback&#34;: &#34;&#34;}
node = {}
if edit is None:
edit = {&#34;host&#34;: True, &#34;protocol&#34;: True, &#34;port&#34;: True, &#34;user&#34;: True, &#34;password&#34;: True, &#34;options&#34;: True, &#34;logs&#34;: True, &#34;tags&#34;: True, &#34;jumphost&#34;: True}
edit = {&#34;host&#34;: True, &#34;protocol&#34;: True, &#34;port&#34;: True, &#34;user&#34;: True, &#34;password&#34;: True, &#34;options&#34;: True, &#34;logs&#34;: True, &#34;tags&#34;: True, &#34;jumphost&#34;: True, &#34;fallback&#34;: True}
questions = []
if edit[&#34;host&#34;]:
questions.append(inquirer.Text(&#34;host&#34;, message=&#34;Add Hostname or IP&#34;, validate=self.validators.host_validation, default=defaults[&#34;host&#34;]))
@@ -561,6 +626,8 @@ el.replaceWith(d);
questions.append(inquirer.List(&#34;password&#34;, message=&#34;Password: Use a local password, no password or a list of profiles to reference?&#34;, choices=[&#34;Local Password&#34;, &#34;Profiles&#34;, &#34;No Password&#34;]))
else:
node[&#34;password&#34;] = defaults[&#34;password&#34;]
if not edit.get(&#34;fallback&#34;):
node[&#34;fallback&#34;] = defaults.get(&#34;fallback&#34;, &#34;&#34;)
answer = inquirer.prompt(questions)
if answer is None:
@@ -578,12 +645,28 @@ el.replaceWith(d);
passa = inquirer.prompt(passq)
if passa is None:
return False
answer[&#34;password&#34;] = passa[&#34;password&#34;].split(&#34;,&#34;)
answer[&#34;password&#34;] = [p.strip() for p in passa[&#34;password&#34;].split(&#34;,&#34;) if p.strip()]
elif answer[&#34;password&#34;] == &#34;No Password&#34;:
answer[&#34;password&#34;] = &#34;&#34;
if &#34;tags&#34; in answer and not answer[&#34;tags&#34;].startswith(&#34;@&#34;) and answer[&#34;tags&#34;]:
answer[&#34;tags&#34;] = ast.literal_eval(answer[&#34;tags&#34;])
if edit.get(&#34;fallback&#34;):
raw_def_fb = defaults.get(&#34;fallback&#34;, &#34;&#34;)
def_fb_str = json.dumps(raw_def_fb) if isinstance(raw_def_fb, (list, dict)) else str(raw_def_fb or &#34;&#34;)
fbq = [
inquirer.Text(
&#34;fallback&#34;,
message=&#34;Add fallback credentials (@profile, @p1,@p2, JSON list/dict, or compose:)&#34;,
validate=self.validators.fallback_validation,
default=def_fb_str.replace(&#34;{&#34;, &#34;{{&#34;).replace(&#34;}&#34;, &#34;}}&#34;),
)
]
fba = inquirer.prompt(fbq)
if fba is None:
return False
answer[&#34;fallback&#34;] = _parse_fallback_input(fba[&#34;fallback&#34;])
if &#34;tags&#34; in answer and answer[&#34;tags&#34;] and not str(answer[&#34;tags&#34;]).startswith(&#34;@&#34;):
answer[&#34;tags&#34;] = _parse_tags_input(answer[&#34;tags&#34;])
result = {**uniques, **answer, **node}
result[&#34;type&#34;] = &#34;connection&#34;
@@ -607,11 +690,13 @@ el.replaceWith(d);
defaults[&#34;tags&#34;] = &#34;&#34;
if &#34;jumphost&#34; not in defaults:
defaults[&#34;jumphost&#34;] = &#34;&#34;
except Exception:
defaults = {&#34;host&#34;: &#34;&#34;, &#34;protocol&#34;: &#34;&#34;, &#34;port&#34;: &#34;&#34;, &#34;user&#34;: &#34;&#34;, &#34;options&#34;: &#34;&#34;, &#34;logs&#34;: &#34;&#34;, &#34;tags&#34;: &#34;&#34;, &#34;jumphost&#34;: &#34;&#34;}
if &#34;fallback&#34; not in defaults:
defaults[&#34;fallback&#34;] = &#34;&#34;
except Exception: # noqa: BLE001
defaults = {&#34;host&#34;: &#34;&#34;, &#34;protocol&#34;: &#34;&#34;, &#34;port&#34;: &#34;&#34;, &#34;user&#34;: &#34;&#34;, &#34;options&#34;: &#34;&#34;, &#34;logs&#34;: &#34;&#34;, &#34;tags&#34;: &#34;&#34;, &#34;jumphost&#34;: &#34;&#34;, &#34;fallback&#34;: &#34;&#34;}
profile = {}
if edit is None:
edit = {&#34;host&#34;: True, &#34;protocol&#34;: True, &#34;port&#34;: True, &#34;user&#34;: True, &#34;password&#34;: True, &#34;options&#34;: True, &#34;logs&#34;: True, &#34;tags&#34;: True, &#34;jumphost&#34;: True}
edit = {&#34;host&#34;: True, &#34;protocol&#34;: True, &#34;port&#34;: True, &#34;user&#34;: True, &#34;password&#34;: True, &#34;options&#34;: True, &#34;logs&#34;: True, &#34;tags&#34;: True, &#34;jumphost&#34;: True, &#34;fallback&#34;: True}
questions = []
if edit[&#34;host&#34;]:
questions.append(inquirer.Text(&#34;host&#34;, message=&#34;Add Hostname or IP&#34;, default=defaults[&#34;host&#34;]))
@@ -649,17 +734,24 @@ el.replaceWith(d);
questions.append(inquirer.Password(&#34;password&#34;, message=&#34;Set Password&#34;))
else:
profile[&#34;password&#34;] = defaults[&#34;password&#34;]
if edit.get(&#34;fallback&#34;):
raw_def_fb = defaults.get(&#34;fallback&#34;, &#34;&#34;)
def_fb_str = json.dumps(raw_def_fb) if isinstance(raw_def_fb, (list, dict)) else str(raw_def_fb or &#34;&#34;)
questions.append(inquirer.Text(&#34;fallback&#34;, message=&#34;Add fallback credentials (@profile, @p1,@p2, JSON list/dict, or compose:)&#34;, validate=self.validators.fallback_validation, default=def_fb_str.replace(&#34;{&#34;, &#34;{{&#34;).replace(&#34;}&#34;, &#34;}}&#34;)))
else:
profile[&#34;fallback&#34;] = defaults.get(&#34;fallback&#34;, &#34;&#34;)
answer = inquirer.prompt(questions)
if answer is None:
return False
if &#34;password&#34; in answer:
if answer[&#34;password&#34;] != &#34;&#34;:
if &#34;password&#34; in answer and answer[&#34;password&#34;] != &#34;&#34;:
answer[&#34;password&#34;] = self.app.services.config_svc.encrypt_password(answer[&#34;password&#34;])
if &#34;tags&#34; in answer and answer[&#34;tags&#34;]:
answer[&#34;tags&#34;] = ast.literal_eval(answer[&#34;tags&#34;])
answer[&#34;tags&#34;] = _parse_tags_input(answer[&#34;tags&#34;])
if &#34;fallback&#34; in answer:
answer[&#34;fallback&#34;] = _parse_fallback_input(answer[&#34;fallback&#34;])
result = {**answer, **profile}
result[&#34;id&#34;] = unique
+2
View File
@@ -193,6 +193,7 @@ el.replaceWith(d);
uniques = self.app.services.nodes.explode_unique(args.data)
if not uniques:
raise InvalidConfigurationError(f&#34;Invalid folder {args.data}&#34;)
self.app.services.nodes.validate_parent_folder(args.data, is_folder=True)
self.app.services.nodes.add_node(args.data, {}, is_folder=True)
printer.success(f&#34;{args.data} added successfully&#34;)
else:
@@ -364,6 +365,7 @@ def forms(self):
uniques = self.app.services.nodes.explode_unique(args.data)
if not uniques:
raise InvalidConfigurationError(f&#34;Invalid folder {args.data}&#34;)
self.app.services.nodes.validate_parent_folder(args.data, is_folder=True)
self.app.services.nodes.add_node(args.data, {}, is_folder=True)
printer.success(f&#34;{args.data} added successfully&#34;)
else:
+8 -2
View File
@@ -210,7 +210,10 @@ el.replaceWith(d);
path = args.data[0]
try:
with open(path, &#34;r&#34;) as f:
playbook = yaml.load(f, Loader=yaml.FullLoader)
playbook = yaml.safe_load(f)
if not isinstance(playbook, dict):
raise TypeError(&#34;Invalid playbook format: expected a dictionary&#34;)
# Check preflight first before any task runs
if getattr(args, &#34;preflight_ai&#34;, False):
@@ -1021,7 +1024,10 @@ el.replaceWith(d);
path = args.data[0]
try:
with open(path, &#34;r&#34;) as f:
playbook = yaml.load(f, Loader=yaml.FullLoader)
playbook = yaml.safe_load(f)
if not isinstance(playbook, dict):
raise TypeError(&#34;Invalid playbook format: expected a dictionary&#34;)
# Check preflight first before any task runs
if getattr(args, &#34;preflight_ai&#34;, False):
File diff suppressed because it is too large. Load diff
+227 -50
View File
@@ -106,50 +106,119 @@ el.replaceWith(d);
return True
def pass_validation(self, answers, current, regex = &#34;(^@.+$)&#34;):
if is_compose_expression(current):
try:
validate_compose_syntax(current)
except SanitizedAuthError as exc:
_raise_val_err(str(exc))
return True
profiles = current.split(&#34;,&#34;)
for i in profiles:
if not re.match(regex, i) or i[1:] not in self.app.profiles:
_raise_val_err(&#34;Profile {} don&#39;t exist&#34;.format(i))
return True
def _validate_tags_dict(self, current: str):
if len(current.encode(&#34;utf-8&#34;, errors=&#34;replace&#34;)) &gt; MAX_JSON_INPUT_BYTES:
_raise_val_err(f&#34;Tags exceed maximum size of {MAX_JSON_INPUT_BYTES} bytes&#34;)
parsed = None
try:
parsed = json.loads(current)
except (ValueError, TypeError):
try:
parsed = ast.literal_eval(current)
except (ValueError, TypeError, SyntaxError, MemoryError, RecursionError):
parsed = None
if not isinstance(parsed, dict):
_raise_val_err(&#34;Tags should be a python dictionary.&#34;)
try:
validate_json_depth_and_size(
parsed,
max_bytes=MAX_JSON_INPUT_BYTES,
max_depth=MAX_JSON_INPUT_DEPTH,
)
validate_auth_budget_tags(parsed)
except (ValueError, TypeError, SanitizedAuthError) as exc:
_raise_val_err(str(exc))
return True
def tags_validation(self, answers, current):
if current.startswith(&#34;@&#34;):
if current[1:] not in self.app.profiles:
_raise_val_err(&#34;Profile {} don&#39;t exist&#34;.format(current))
elif current != &#34;&#34;:
isdict = False
try:
isdict = ast.literal_eval(current)
except Exception:
pass
if not isinstance (isdict, dict):
_raise_val_err(&#34;Tags should be a python dictionary.&#34;.format(current))
return self._validate_tags_dict(current)
return True
def profile_tags_validation(self, answers, current):
if current != &#34;&#34;:
isdict = False
try:
isdict = ast.literal_eval(current)
except Exception:
pass
if not isinstance (isdict, dict):
_raise_val_err(&#34;Tags should be a python dictionary.&#34;.format(current))
return self._validate_tags_dict(current)
return True
def fallback_validation(self, answers, current):
if current in (None, &#34;&#34;):
return True
stripped = str(current).strip()
if not stripped:
return True
if len(stripped.encode(&#34;utf-8&#34;, errors=&#34;replace&#34;)) &gt; MAX_JSON_INPUT_BYTES:
_raise_val_err(f&#34;Fallback exceeds maximum size of {MAX_JSON_INPUT_BYTES} bytes&#34;)
if stripped.startswith(&#34;@&#34;) and not stripped.startswith(&#34;@@&#34;) and &#34;+&#34; not in stripped:
parts = [p.strip() for p in stripped.split(&#34;,&#34;)]
if any(not p for p in parts):
_raise_val_err(&#34;Fallback profile entry cannot be empty&#34;)
if len(parts) &gt; MAX_FALLBACK_ENTRIES:
_raise_val_err(f&#34;Fallback exceeds maximum of {MAX_FALLBACK_ENTRIES} entries&#34;)
for p in parts:
if not p.startswith(&#34;@&#34;) or not re.fullmatch(r&#34;[A-Za-z0-9_.$:/-]+&#34;, p[1:]):
_raise_val_err(
&#34;Comma-separated fallback entries must be valid @profile references&#34;
)
if p[1:] not in self.app.profiles:
_raise_val_err(f&#34;Profile {p} don&#39;t exist&#34;)
return True
try:
if is_compose_expression(stripped):
validate_compose_syntax(stripped)
return True
if stripped.startswith((&#34;[&#34;, &#34;{&#34;)):
validate_json_depth_and_size(
stripped,
max_bytes=MAX_JSON_INPUT_BYTES,
max_depth=MAX_JSON_INPUT_DEPTH,
)
validate_fallback_entries(stripped, max_entries=MAX_FALLBACK_ENTRIES)
except (ValueError, TypeError, SanitizedAuthError) as exc:
_raise_val_err(str(exc))
return True
def _get_all_valid_nodes(self) -&gt; list[str]:
valid_nodes = list(getattr(self.app, &#34;nodes_list&#34;, []) or [])
if hasattr(self.app, &#34;services&#34;) and hasattr(self.app.services, &#34;nodes&#34;):
raw_fn = getattr(self.app.services.nodes, &#34;_get_raw_nodes&#34;, None)
if callable(raw_fn) and not hasattr(raw_fn, &#34;assert_called&#34;):
raw_res = raw_fn()
if isinstance(raw_res, (list, tuple, set)):
valid_nodes = list(dict.fromkeys([*raw_res, *valid_nodes]))
return valid_nodes
def _node_exists(self, target: str) -&gt; bool:
valid_nodes = self._get_all_valid_nodes()
if getattr(self.app, &#34;case&#34;, True):
return target in valid_nodes
return target.lower() in {n.lower() for n in valid_nodes}
def jumphost_validation(self, answers, current):
if current.startswith(&#34;@&#34;):
if current[1:] not in self.app.profiles:
_raise_val_err(&#34;Profile {} don&#39;t exist&#34;.format(current))
elif current != &#34;&#34;:
if current not in self.app.nodes_list:
_raise_val_err(&#34;Node {} don&#39;t exist.&#34;.format(current))
_raise_val_err(f&#34;Profile {current} don&#39;t exist&#34;)
elif current != &#34;&#34; and not self._node_exists(current):
_raise_val_err(f&#34;Node {current} don&#39;t exist.&#34;)
return True
def profile_jumphost_validation(self, answers, current):
if current != &#34;&#34;:
if current not in self.app.nodes_list:
_raise_val_err(&#34;Node {} don&#39;t exist.&#34;.format(current))
if current != &#34;&#34; and not self._node_exists(current):
_raise_val_err(f&#34;Node {current} don&#39;t exist.&#34;)
return True
def default_validation(self, answers, current):
@@ -167,12 +236,46 @@ el.replaceWith(d);
return True
def bulk_folder_validation(self, answers, current):
if not self.app.case:
case = getattr(self.app, &#34;case&#34;, False)
if not case:
current = current.lower()
candidate = current
if &#34;/&#34; in current:
candidate = current.split(&#34;/&#34;)[0]
candidate = current.split(&#34;/&#34;)[0] if &#34;/&#34; in current else current
if hasattr(self.app, &#34;services&#34;) and hasattr(self.app.services, &#34;context&#34;):
raw_folders = list(getattr(self.app, &#34;folders&#34;, []) or [])
if hasattr(self.app.services, &#34;nodes&#34;):
raw_fn = getattr(self.app.services.nodes, &#34;_get_raw_folders&#34;, None)
if callable(raw_fn) and not hasattr(raw_fn, &#34;assert_called&#34;):
raw_res = raw_fn()
if isinstance(raw_res, (list, tuple, set)):
raw_folders = list(raw_res)
def _folder_exists_norm(folder_name: str) -&gt; bool:
if case:
return folder_name in raw_folders
return folder_name.lower() in {f.lower() for f in raw_folders}
if current != &#34;&#34; and not _folder_exists_norm(candidate):
_raise_val_err(f&#34;Location {current} don&#39;t exist&#34;)
ctx_svc = self.app.services.context
if ctx_svc is not None and not ctx_svc.is_default_all_context():
raw_ids = answers.get(&#34;ids&#34;, &#34;&#34;) if isinstance(answers, dict) else &#34;&#34;
for nid in str(raw_ids).split(&#34;,&#34;):
nid_stripped = nid.strip()
if not nid_stripped:
continue
nid_norm = nid_stripped if case else nid_stripped.lower()
full_id = f&#34;{nid_norm}{candidate}&#34; if candidate else nid_norm
if not ctx_svc.matches_active_context(full_id):
_raise_val_err(
f&#34;Node &#39;{full_id}&#39; does not match active context &#39;{ctx_svc.current_context}&#39;.&#34;
)
parent = full_id.partition(&#34;@&#34;)[2]
if parent and not _folder_exists_norm(f&#34;@{parent}&#34;):
_raise_val_err(f&#34;Location @{parent} don&#39;t exist&#34;)
return True
matches = list(filter(lambda k: k == candidate, self.app.folders))
if current != &#34;&#34; and len(matches) == 0:
@@ -203,12 +306,46 @@ el.replaceWith(d);
<span>Expand source code</span>
</summary>
<pre><code class="python">def bulk_folder_validation(self, answers, current):
if not self.app.case:
case = getattr(self.app, &#34;case&#34;, False)
if not case:
current = current.lower()
candidate = current
if &#34;/&#34; in current:
candidate = current.split(&#34;/&#34;)[0]
candidate = current.split(&#34;/&#34;)[0] if &#34;/&#34; in current else current
if hasattr(self.app, &#34;services&#34;) and hasattr(self.app.services, &#34;context&#34;):
raw_folders = list(getattr(self.app, &#34;folders&#34;, []) or [])
if hasattr(self.app.services, &#34;nodes&#34;):
raw_fn = getattr(self.app.services.nodes, &#34;_get_raw_folders&#34;, None)
if callable(raw_fn) and not hasattr(raw_fn, &#34;assert_called&#34;):
raw_res = raw_fn()
if isinstance(raw_res, (list, tuple, set)):
raw_folders = list(raw_res)
def _folder_exists_norm(folder_name: str) -&gt; bool:
if case:
return folder_name in raw_folders
return folder_name.lower() in {f.lower() for f in raw_folders}
if current != &#34;&#34; and not _folder_exists_norm(candidate):
_raise_val_err(f&#34;Location {current} don&#39;t exist&#34;)
ctx_svc = self.app.services.context
if ctx_svc is not None and not ctx_svc.is_default_all_context():
raw_ids = answers.get(&#34;ids&#34;, &#34;&#34;) if isinstance(answers, dict) else &#34;&#34;
for nid in str(raw_ids).split(&#34;,&#34;):
nid_stripped = nid.strip()
if not nid_stripped:
continue
nid_norm = nid_stripped if case else nid_stripped.lower()
full_id = f&#34;{nid_norm}{candidate}&#34; if candidate else nid_norm
if not ctx_svc.matches_active_context(full_id):
_raise_val_err(
f&#34;Node &#39;{full_id}&#39; does not match active context &#39;{ctx_svc.current_context}&#39;.&#34;
)
parent = full_id.partition(&#34;@&#34;)[2]
if parent and not _folder_exists_norm(f&#34;@{parent}&#34;):
_raise_val_err(f&#34;Location @{parent} don&#39;t exist&#34;)
return True
matches = list(filter(lambda k: k == candidate, self.app.folders))
if current != &#34;&#34; and len(matches) == 0:
@@ -273,6 +410,53 @@ el.replaceWith(d);
</details>
<div class="desc"></div>
</dd>
<dt id="connpy.cli.validators.Validators.fallback_validation"><code class="name flex">
<span>def <span class="ident">fallback_validation</span></span>(<span>self, answers, current)</span>
</code></dt>
<dd>
<details class="source">
<summary>
<span>Expand source code</span>
</summary>
<pre><code class="python">def fallback_validation(self, answers, current):
if current in (None, &#34;&#34;):
return True
stripped = str(current).strip()
if not stripped:
return True
if len(stripped.encode(&#34;utf-8&#34;, errors=&#34;replace&#34;)) &gt; MAX_JSON_INPUT_BYTES:
_raise_val_err(f&#34;Fallback exceeds maximum size of {MAX_JSON_INPUT_BYTES} bytes&#34;)
if stripped.startswith(&#34;@&#34;) and not stripped.startswith(&#34;@@&#34;) and &#34;+&#34; not in stripped:
parts = [p.strip() for p in stripped.split(&#34;,&#34;)]
if any(not p for p in parts):
_raise_val_err(&#34;Fallback profile entry cannot be empty&#34;)
if len(parts) &gt; MAX_FALLBACK_ENTRIES:
_raise_val_err(f&#34;Fallback exceeds maximum of {MAX_FALLBACK_ENTRIES} entries&#34;)
for p in parts:
if not p.startswith(&#34;@&#34;) or not re.fullmatch(r&#34;[A-Za-z0-9_.$:/-]+&#34;, p[1:]):
_raise_val_err(
&#34;Comma-separated fallback entries must be valid @profile references&#34;
)
if p[1:] not in self.app.profiles:
_raise_val_err(f&#34;Profile {p} don&#39;t exist&#34;)
return True
try:
if is_compose_expression(stripped):
validate_compose_syntax(stripped)
return True
if stripped.startswith((&#34;[&#34;, &#34;{&#34;)):
validate_json_depth_and_size(
stripped,
max_bytes=MAX_JSON_INPUT_BYTES,
max_depth=MAX_JSON_INPUT_DEPTH,
)
validate_fallback_entries(stripped, max_entries=MAX_FALLBACK_ENTRIES)
except (ValueError, TypeError, SanitizedAuthError) as exc:
_raise_val_err(str(exc))
return True</code></pre>
</details>
<div class="desc"></div>
</dd>
<dt id="connpy.cli.validators.Validators.host_validation"><code class="name flex">
<span>def <span class="ident">host_validation</span></span>(<span>self, answers, current, regex='^.+$')</span>
</code></dt>
@@ -302,10 +486,9 @@ el.replaceWith(d);
<pre><code class="python">def jumphost_validation(self, answers, current):
if current.startswith(&#34;@&#34;):
if current[1:] not in self.app.profiles:
_raise_val_err(&#34;Profile {} don&#39;t exist&#34;.format(current))
elif current != &#34;&#34;:
if current not in self.app.nodes_list:
_raise_val_err(&#34;Node {} don&#39;t exist.&#34;.format(current))
_raise_val_err(f&#34;Profile {current} don&#39;t exist&#34;)
elif current != &#34;&#34; and not self._node_exists(current):
_raise_val_err(f&#34;Node {current} don&#39;t exist.&#34;)
return True</code></pre>
</details>
<div class="desc"></div>
@@ -319,6 +502,12 @@ el.replaceWith(d);
<span>Expand source code</span>
</summary>
<pre><code class="python">def pass_validation(self, answers, current, regex = &#34;(^@.+$)&#34;):
if is_compose_expression(current):
try:
validate_compose_syntax(current)
except SanitizedAuthError as exc:
_raise_val_err(str(exc))
return True
profiles = current.split(&#34;,&#34;)
for i in profiles:
if not re.match(regex, i) or i[1:] not in self.app.profiles:
@@ -360,9 +549,8 @@ el.replaceWith(d);
<span>Expand source code</span>
</summary>
<pre><code class="python">def profile_jumphost_validation(self, answers, current):
if current != &#34;&#34;:
if current not in self.app.nodes_list:
_raise_val_err(&#34;Node {} don&#39;t exist.&#34;.format(current))
if current != &#34;&#34; and not self._node_exists(current):
_raise_val_err(f&#34;Node {current} don&#39;t exist.&#34;)
return True</code></pre>
</details>
<div class="desc"></div>
@@ -413,13 +601,7 @@ el.replaceWith(d);
</summary>
<pre><code class="python">def profile_tags_validation(self, answers, current):
if current != &#34;&#34;:
isdict = False
try:
isdict = ast.literal_eval(current)
except Exception:
pass
if not isinstance (isdict, dict):
_raise_val_err(&#34;Tags should be a python dictionary.&#34;.format(current))
return self._validate_tags_dict(current)
return True</code></pre>
</details>
<div class="desc"></div>
@@ -455,13 +637,7 @@ el.replaceWith(d);
if current[1:] not in self.app.profiles:
_raise_val_err(&#34;Profile {} don&#39;t exist&#34;.format(current))
elif current != &#34;&#34;:
isdict = False
try:
isdict = ast.literal_eval(current)
except Exception:
pass
if not isinstance (isdict, dict):
_raise_val_err(&#34;Tags should be a python dictionary.&#34;.format(current))
return self._validate_tags_dict(current)
return True</code></pre>
</details>
<div class="desc"></div>
@@ -490,6 +666,7 @@ el.replaceWith(d);
<li><code><a title="connpy.cli.validators.Validators.bulk_host_validation" href="#connpy.cli.validators.Validators.bulk_host_validation">bulk_host_validation</a></code></li>
<li><code><a title="connpy.cli.validators.Validators.bulk_node_validation" href="#connpy.cli.validators.Validators.bulk_node_validation">bulk_node_validation</a></code></li>
<li><code><a title="connpy.cli.validators.Validators.default_validation" href="#connpy.cli.validators.Validators.default_validation">default_validation</a></code></li>
<li><code><a title="connpy.cli.validators.Validators.fallback_validation" href="#connpy.cli.validators.Validators.fallback_validation">fallback_validation</a></code></li>
<li><code><a title="connpy.cli.validators.Validators.host_validation" href="#connpy.cli.validators.Validators.host_validation">host_validation</a></code></li>
<li><code><a title="connpy.cli.validators.Validators.jumphost_validation" href="#connpy.cli.validators.Validators.jumphost_validation">jumphost_validation</a></code></li>
<li><code><a title="connpy.cli.validators.Validators.pass_validation" href="#connpy.cli.validators.Validators.pass_validation">pass_validation</a></code></li>
+18 -9
View File
@@ -1671,8 +1671,9 @@ def predict_execution_results(request,
</details>
<div class="desc"><p>Missing associated documentation comment in .proto file.</p>
<p>Constructor.</p>
<h2 id="args">Args</h2>
<dl>
<dt>Args</dt>
<dt>-----=</dt>
<dt><strong><code>channel</code></strong></dt>
<dd>A grpc.Channel.</dd>
</dl></div>
@@ -2371,8 +2372,9 @@ def revoke_api_token(request,
</details>
<div class="desc"><p>Missing associated documentation comment in .proto file.</p>
<p>Constructor.</p>
<h2 id="args">Args</h2>
<dl>
<dt>Args</dt>
<dt>-----=</dt>
<dt><strong><code>channel</code></strong></dt>
<dd>A grpc.Channel.</dd>
</dl></div>
@@ -2980,8 +2982,9 @@ def update_setting(request,
</details>
<div class="desc"><p>Missing associated documentation comment in .proto file.</p>
<p>Constructor.</p>
<h2 id="args">Args</h2>
<dl>
<dt>Args</dt>
<dt>-----=</dt>
<dt><strong><code>channel</code></strong></dt>
<dd>A grpc.Channel.</dd>
</dl></div>
@@ -3316,8 +3319,9 @@ def test_commands(request,
</details>
<div class="desc"><p>Missing associated documentation comment in .proto file.</p>
<p>Constructor.</p>
<h2 id="args">Args</h2>
<dl>
<dt>Args</dt>
<dt>-----=</dt>
<dt><strong><code>channel</code></strong></dt>
<dd>A grpc.Channel.</dd>
</dl></div>
@@ -3652,8 +3656,9 @@ def set_reserved_names(request,
</details>
<div class="desc"><p>Missing associated documentation comment in .proto file.</p>
<p>Constructor.</p>
<h2 id="args">Args</h2>
<dl>
<dt>Args</dt>
<dt>-----=</dt>
<dt><strong><code>channel</code></strong></dt>
<dd>A grpc.Channel.</dd>
</dl></div>
@@ -5080,8 +5085,9 @@ def validate_parent_folder(request,
</details>
<div class="desc"><p>Missing associated documentation comment in .proto file.</p>
<p>Constructor.</p>
<h2 id="args">Args</h2>
<dl>
<dt>Args</dt>
<dt>-----=</dt>
<dt><strong><code>channel</code></strong></dt>
<dd>A grpc.Channel.</dd>
</dl></div>
@@ -5598,8 +5604,9 @@ def list_plugins(request,
</details>
<div class="desc"><p>Missing associated documentation comment in .proto file.</p>
<p>Constructor.</p>
<h2 id="args">Args</h2>
<dl>
<dt>Args</dt>
<dt>-----=</dt>
<dt><strong><code>channel</code></strong></dt>
<dd>A grpc.Channel.</dd>
</dl></div>
@@ -6207,8 +6214,9 @@ def update_profile(request,
</details>
<div class="desc"><p>Missing associated documentation comment in .proto file.</p>
<p>Constructor.</p>
<h2 id="args">Args</h2>
<dl>
<dt>Args</dt>
<dt>-----=</dt>
<dt><strong><code>channel</code></strong></dt>
<dd>A grpc.Channel.</dd>
</dl></div>
@@ -6725,8 +6733,9 @@ def stop_api(request,
</details>
<div class="desc"><p>Missing associated documentation comment in .proto file.</p>
<p>Constructor.</p>
<h2 id="args">Args</h2>
<dl>
<dt>Args</dt>
<dt>-----=</dt>
<dt><strong><code>channel</code></strong></dt>
<dd>A grpc.Channel.</dd>
</dl></div>
@@ -62,7 +62,7 @@ el.replaceWith(d);
<dl>
<dt id="connpy.grpc_layer.remote_plugin_pb2.IdRequest.DESCRIPTOR"><code class="name">var <span class="ident">DESCRIPTOR</span></code></dt>
<dd>
<div class="desc"><p>The type of the None singleton.</p></div>
<div class="desc"></div>
</dd>
</dl>
</dd>
@@ -81,7 +81,7 @@ el.replaceWith(d);
<dl>
<dt id="connpy.grpc_layer.remote_plugin_pb2.OutputChunk.DESCRIPTOR"><code class="name">var <span class="ident">DESCRIPTOR</span></code></dt>
<dd>
<div class="desc"><p>The type of the None singleton.</p></div>
<div class="desc"></div>
</dd>
</dl>
</dd>
@@ -100,7 +100,7 @@ el.replaceWith(d);
<dl>
<dt id="connpy.grpc_layer.remote_plugin_pb2.PluginInvokeRequest.DESCRIPTOR"><code class="name">var <span class="ident">DESCRIPTOR</span></code></dt>
<dd>
<div class="desc"><p>The type of the None singleton.</p></div>
<div class="desc"></div>
</dd>
</dl>
</dd>
@@ -119,7 +119,7 @@ el.replaceWith(d);
<dl>
<dt id="connpy.grpc_layer.remote_plugin_pb2.StringResponse.DESCRIPTOR"><code class="name">var <span class="ident">DESCRIPTOR</span></code></dt>
<dd>
<div class="desc"><p>The type of the None singleton.</p></div>
<div class="desc"></div>
</dd>
</dl>
</dd>
@@ -317,8 +317,9 @@ def invoke_plugin(request,
</details>
<div class="desc"><p>Missing associated documentation comment in .proto file.</p>
<p>Constructor.</p>
<h2 id="args">Args</h2>
<dl>
<dt>Args</dt>
<dt>-----=</dt>
<dt><strong><code>channel</code></strong></dt>
<dd>A grpc.Channel.</dd>
</dl></div>
+77 -51
View File
@@ -55,6 +55,7 @@ el.replaceWith(d);
<pre><code class="python">def handle_errors(func):
import inspect
if inspect.isgeneratorfunction(func):
@functools.wraps(func)
def wrapper(*args, **kwargs):
try:
for item in func(*args, **kwargs):
@@ -75,6 +76,7 @@ el.replaceWith(d);
printer.clear_thread_state()
return wrapper
else:
@functools.wraps(func)
def wrapper(*args, **kwargs):
try:
return func(*args, **kwargs)
@@ -108,7 +110,7 @@ el.replaceWith(d);
from connpy.grpc_layer.user_registry import UserRegistry
from connpy.services.provider import ServiceProvider
fallback_provider = ServiceProvider(config, mode=&#34;local&#34;)
fallback_provider = ServiceProvider(config, mode=&#34;local&#34;, enforce_context=False)
registry = UserRegistry(config.defaultdir)
# Check if trusted_gateway provider is configured if SSO Gateway Secret is present in env
@@ -161,9 +163,7 @@ el.replaceWith(d);
</summary>
<pre><code class="python">class AIServicer(connpy_pb2_grpc.AIServiceServicer):
def __init__(self, provider, registry=None, debug=False):
if not hasattr(provider, &#34;mode&#34;):
from connpy.services.provider import ServiceProvider
provider = ServiceProvider(provider, mode=&#34;local&#34;)
provider = _ensure_server_provider(provider)
self._fallback_provider = provider
self._registry = registry
self.server_debug = debug
@@ -604,7 +604,7 @@ def service(self):
<dl>
<dt id="connpy.grpc_layer.server.AuthInterceptor.OPEN_METHODS"><code class="name">var <span class="ident">OPEN_METHODS</span></code></dt>
<dd>
<div class="desc"><p>The type of the None singleton.</p></div>
<div class="desc"></div>
</dd>
</dl>
<h3>Methods</h3>
@@ -648,8 +648,9 @@ contextvars.Context, and the same Context is used for downstream
interceptors and for the final handler call. Note that there are no
guarantees that interceptors and handlers will be called from the
same thread.</p>
<h2 id="args">Args</h2>
<dl>
<dt>Args</dt>
<dt>-----=</dt>
<dt><strong><code>continuation</code></strong></dt>
<dd>A function that takes a HandlerCallDetails and
proceeds to invoke the next interceptor in the chain, if any,
@@ -659,8 +660,9 @@ the RPC is considered serviced, or None otherwise.</dd>
<dt><strong><code>handler_call_details</code></strong></dt>
<dd>A HandlerCallDetails describing the RPC.</dd>
</dl>
<h2 id="returns">Returns</h2>
<p>An RpcMethodHandler with which the RPC may be serviced if the
<p>Returns
-----=
An RpcMethodHandler with which the RPC may be serviced if the
interceptor chooses to service this RPC, or None otherwise.</p></div>
</dd>
</dl>
@@ -923,9 +925,7 @@ interceptor chooses to service this RPC, or None otherwise.</p></div>
</summary>
<pre><code class="python">class ConfigServicer(connpy_pb2_grpc.ConfigServiceServicer):
def __init__(self, provider, registry=None):
if not hasattr(provider, &#34;mode&#34;):
from connpy.services.provider import ServiceProvider
provider = ServiceProvider(provider, mode=&#34;local&#34;)
provider = _ensure_server_provider(provider)
self._fallback_provider = provider
self._registry = registry
@@ -1011,9 +1011,7 @@ def service(self):
</summary>
<pre><code class="python">class ExecutionServicer(connpy_pb2_grpc.ExecutionServiceServicer):
def __init__(self, provider, registry=None):
if not hasattr(provider, &#34;mode&#34;):
from connpy.services.provider import ServiceProvider
provider = ServiceProvider(provider, mode=&#34;local&#34;)
provider = _ensure_server_provider(provider)
self._fallback_provider = provider
self._registry = registry
@@ -1182,9 +1180,7 @@ def service(self):
</summary>
<pre><code class="python">class ImportExportServicer(connpy_pb2_grpc.ImportExportServiceServicer):
def __init__(self, provider, registry=None):
if not hasattr(provider, &#34;mode&#34;):
from connpy.services.provider import ServiceProvider
provider = ServiceProvider(provider, mode=&#34;local&#34;)
provider = _ensure_server_provider(provider)
self._fallback_provider = provider
self._registry = registry
@@ -1213,7 +1209,7 @@ def service(self):
if request.value.startswith(&#34;---YAML---\n&#34;):
import yaml
content = request.value[len(&#34;---YAML---\n&#34;):]
data = yaml.load(content, Loader=yaml.FullLoader)
data = yaml.safe_load(content)
self.service.import_from_dict(data)
else:
self.service.import_from_file(request.value)
@@ -1340,8 +1336,9 @@ contextvars.Context, and the same Context is used for downstream
interceptors and for the final handler call. Note that there are no
guarantees that interceptors and handlers will be called from the
same thread.</p>
<h2 id="args">Args</h2>
<dl>
<dt>Args</dt>
<dt>-----=</dt>
<dt><strong><code>continuation</code></strong></dt>
<dd>A function that takes a HandlerCallDetails and
proceeds to invoke the next interceptor in the chain, if any,
@@ -1351,8 +1348,9 @@ the RPC is considered serviced, or None otherwise.</dd>
<dt><strong><code>handler_call_details</code></strong></dt>
<dd>A HandlerCallDetails describing the RPC.</dd>
</dl>
<h2 id="returns">Returns</h2>
<p>An RpcMethodHandler with which the RPC may be serviced if the
<p>Returns
-----=
An RpcMethodHandler with which the RPC may be serviced if the
interceptor chooses to service this RPC, or None otherwise.</p></div>
</dd>
</dl>
@@ -1368,9 +1366,7 @@ interceptor chooses to service this RPC, or None otherwise.</p></div>
</summary>
<pre><code class="python">class NodeServicer(connpy_pb2_grpc.NodeServiceServicer):
def __init__(self, provider, registry=None, debug=False):
if not hasattr(provider, &#34;mode&#34;):
from connpy.services.provider import ServiceProvider
provider = ServiceProvider(provider, mode=&#34;local&#34;)
provider = _ensure_server_provider(provider)
self._fallback_provider = provider
self._registry = registry
self.server_debug = debug
@@ -1421,8 +1417,24 @@ interceptor chooses to service this RPC, or None otherwise.</p></div>
self.server_console.print(f&#34;[debug][DEBUG][/debug] gRPC interact_node request for: [bold cyan]{unique_id}[/bold cyan]&#34;)
if first_req.connection_params_json:
import json
params = json.loads(first_req.connection_params_json)
try:
params = validate_json_depth_and_size(
first_req.connection_params_json,
max_bytes=MAX_JSON_INPUT_BYTES,
max_depth=MAX_JSON_INPUT_DEPTH,
)
if not isinstance(params, dict):
raise TypeError(&#34;connection_params_json must be a JSON object&#34;)
if &#34;fallback&#34; in params and params[&#34;fallback&#34;] not in (None, &#34;&#34;, []):
validate_fallback_entries(
params[&#34;fallback&#34;],
max_entries=MAX_FALLBACK_ENTRIES,
)
if &#34;tags&#34; in params and params[&#34;tags&#34;] not in (&#34;&#34;, None):
validate_auth_budget_tags(params[&#34;tags&#34;])
except (ValueError, TypeError, SanitizedAuthError) as exc:
context.abort(grpc.StatusCode.INVALID_ARGUMENT, str(exc))
base_node_id = params.get(&#34;base_node&#34;)
# Valid attributes that a node object accepts
valid_attrs = [&#39;host&#39;, &#39;options&#39;, &#39;logs&#39;, &#39;password&#39;, &#39;port&#39;, &#39;protocol&#39;, &#39;user&#39;, &#39;jumphost&#39;]
@@ -1440,6 +1452,8 @@ interceptor chooses to service this RPC, or None otherwise.</p></div>
for attr in valid_attrs:
if attr in params:
device[attr] = params[attr]
if &#34;fallback&#34; in params:
device[&#34;fallback&#34;] = params[&#34;fallback&#34;]
if &#34;tags&#34; in params:
device_tags = device.get(&#34;tags&#34;, {})
@@ -1453,20 +1467,30 @@ interceptor chooses to service this RPC, or None otherwise.</p></div>
else:
# base_node not found, fall back to dynamic
node_name = params.get(&#34;name&#34;, fallback_id)
n = node(node_name, host=params.get(&#34;host&#34;, &#34;&#34;), config=user_config)
for attr in valid_attrs:
if attr in params:
setattr(n, attr, params[attr])
dyn_kwargs = {k: params[k] for k in valid_attrs if k in params and k != &#34;host&#34;}
if &#34;fallback&#34; in params:
dyn_kwargs[&#34;fallback&#34;] = params[&#34;fallback&#34;]
if &#34;tags&#34; in params:
n.tags = params[&#34;tags&#34;]
dyn_kwargs[&#34;tags&#34;] = params[&#34;tags&#34;]
n = node(
node_name,
host=params.get(&#34;host&#34;, &#34;&#34;),
config=user_config,
**dyn_kwargs,
)
else:
node_name = params.get(&#34;name&#34;, fallback_id)
n = node(node_name, host=params.get(&#34;host&#34;, &#34;&#34;), config=user_config)
for attr in valid_attrs:
if attr in params:
setattr(n, attr, params[attr])
dyn_kwargs = {k: params[k] for k in valid_attrs if k in params and k != &#34;host&#34;}
if &#34;fallback&#34; in params:
dyn_kwargs[&#34;fallback&#34;] = params[&#34;fallback&#34;]
if &#34;tags&#34; in params:
n.tags = params[&#34;tags&#34;]
dyn_kwargs[&#34;tags&#34;] = params[&#34;tags&#34;]
n = node(
node_name,
host=params.get(&#34;host&#34;, &#34;&#34;),
config=user_config,
**dyn_kwargs,
)
n._orig_protocol = n.protocol
if sftp:
n.protocol = &#34;sftp&#34;
@@ -1487,9 +1511,11 @@ interceptor chooses to service this RPC, or None otherwise.</p></div>
debug_chunks = []
if debug:
from io import StringIO
from rich.console import Console as RichConsole
from ..printer import connpy_theme
from .. import printer as _printer
from ..printer import connpy_theme
def remote_logger(msg_type, message):
buf = StringIO()
@@ -1510,7 +1536,14 @@ interceptor chooses to service this RPC, or None otherwise.</p></div>
else:
remote_logger = None
connect = n._connect(debug=debug, logger=remote_logger)
try:
connect = n._connect(
debug=debug,
logger=remote_logger,
attempt_tracker=getattr(provider, &#34;auth_tracker&#34;, None),
)
except Exception as exc: # noqa: BLE001
connect = str(sanitize_exception(exc, code=&#34;AUTH_FAILED&#34;))
# Send debug output to client before checking result (always show the command)
for chunk in debug_chunks:
@@ -1841,10 +1874,9 @@ interceptor chooses to service this RPC, or None otherwise.</p></div>
continue
asyncio.run(n._async_interact_loop(remote_stream, resize_callback, copilot_handler=remote_copilot_handler))
except Exception as e:
import traceback
print(f&#34;[ERROR in run_async_loop] {e}&#34;)
traceback.print_exc()
except Exception as e: # noqa: BLE001
sanitized_err = sanitize_exception(e, code=&#34;TRANSPORT_ERROR&#34;, context_label=&#34;interact stream&#34;)
print(f&#34;[ERROR in run_async_loop] {sanitized_err}&#34;)
finally:
n._teardown_interact_environment()
response_queue.put(None) # Signal EOF
@@ -1999,9 +2031,7 @@ def service(self):
</summary>
<pre><code class="python">class PluginServicer(connpy_pb2_grpc.PluginServiceServicer, remote_plugin_pb2_grpc.RemotePluginServiceServicer):
def __init__(self, provider, registry=None):
if not hasattr(provider, &#34;mode&#34;):
from connpy.services.provider import ServiceProvider
provider = ServiceProvider(provider, mode=&#34;local&#34;)
provider = _ensure_server_provider(provider)
self._fallback_provider = provider
self._registry = registry
@@ -2106,9 +2136,7 @@ def service(self):
</summary>
<pre><code class="python">class ProfileServicer(connpy_pb2_grpc.ProfileServiceServicer):
def __init__(self, provider, registry=None):
if not hasattr(provider, &#34;mode&#34;):
from connpy.services.provider import ServiceProvider
provider = ServiceProvider(provider, mode=&#34;local&#34;)
provider = _ensure_server_provider(provider)
self._fallback_provider = provider
self._registry = registry
@@ -2409,9 +2437,7 @@ def service(self):
</summary>
<pre><code class="python">class SystemServicer(connpy_pb2_grpc.SystemServiceServicer):
def __init__(self, provider, registry=None):
if not hasattr(provider, &#34;mode&#34;):
from connpy.services.provider import ServiceProvider
provider = ServiceProvider(provider, mode=&#34;local&#34;)
provider = _ensure_server_provider(provider)
self._fallback_provider = provider
self._registry = registry
+223 -42
View File
@@ -687,8 +687,9 @@ def predict_execution_results(self, target_nodes, commands, status=None, chunk_c
return continuation(new_details, request_iterator)</code></pre>
</details>
<div class="desc"><p>Intercepts a stream-stream invocation.</p>
<h2 id="args">Args</h2>
<dl>
<dt>Args</dt>
<dt>-----=</dt>
<dt><strong><code>continuation</code></strong></dt>
<dd>A function that proceeds with the invocation by
executing the next interceptor in chain or invoking the
@@ -707,8 +708,9 @@ outgoing RPC.</dd>
<dt><strong><code>request_iterator</code></strong></dt>
<dd>An iterator that yields request values for the RPC.</dd>
</dl>
<h2 id="returns">Returns</h2>
<p>An object that is both a Call for the RPC and an iterator of
<p>Returns
-----=
An object that is both a Call for the RPC and an iterator of
response values. Drawing response values from the returned
Call-iterator may raise RpcError indicating termination of
the RPC with non-OK status. This object <em>should</em> also fulfill the
@@ -727,8 +729,9 @@ Future interface, though it may not.</p></div>
return continuation(new_details, request_iterator)</code></pre>
</details>
<div class="desc"><p>Intercepts a stream-unary invocation asynchronously.</p>
<h2 id="args">Args</h2>
<dl>
<dt>Args</dt>
<dt>-----=</dt>
<dt><strong><code>continuation</code></strong></dt>
<dd>A function that proceeds with the invocation by
executing the next interceptor in chain or invoking the
@@ -747,8 +750,9 @@ outgoing RPC.</dd>
<dt><strong><code>request_iterator</code></strong></dt>
<dd>An iterator that yields request values for the RPC.</dd>
</dl>
<h2 id="returns">Returns</h2>
<p>An object that is both a Call for the RPC and a Future.
<p>Returns
-----=
An object that is both a Call for the RPC and a Future.
In the event of RPC completion, the return Call-Future's
result value will be the response message of the RPC.
Should the event terminate with non-OK status, the returned
@@ -767,8 +771,9 @@ Call-Future's exception value will be an RpcError.</p></div>
return continuation(new_details, request)</code></pre>
</details>
<div class="desc"><p>Intercepts a unary-stream invocation.</p>
<h2 id="args">Args</h2>
<dl>
<dt>Args</dt>
<dt>-----=</dt>
<dt><strong><code>continuation</code></strong></dt>
<dd>A function that proceeds with the invocation by
executing the next interceptor in chain or invoking the
@@ -787,8 +792,9 @@ outgoing RPC.</dd>
<dt><strong><code>request</code></strong></dt>
<dd>The request value for the RPC.</dd>
</dl>
<h2 id="returns">Returns</h2>
<p>An object that is both a Call for the RPC and an iterator of
<p>Returns
-----=
An object that is both a Call for the RPC and an iterator of
response values. Drawing response values from the returned
Call-iterator may raise RpcError indicating termination of
the RPC with non-OK status. This object <em>should</em> also fulfill the
@@ -807,8 +813,9 @@ Future interface, though it may not.</p></div>
return continuation(new_details, request)</code></pre>
</details>
<div class="desc"><p>Intercepts a unary-unary invocation asynchronously.</p>
<h2 id="args">Args</h2>
<dl>
<dt>Args</dt>
<dt>-----=</dt>
<dt><strong><code>continuation</code></strong></dt>
<dd>A function that proceeds with the invocation by
executing the next interceptor in chain or invoking the
@@ -828,8 +835,9 @@ outgoing RPC.</dd>
<dt><strong><code>request</code></strong></dt>
<dd>The request value for the RPC.</dd>
</dl>
<h2 id="returns">Returns</h2>
<p>An object that is both a Call for the RPC and a Future.
<p>Returns
-----=
An object that is both a Call for the RPC and a Future.
In the event of RPC completion, the return Call-Future's
result value will be the response message of the RPC.
Should the event terminate with non-OK status, the returned
@@ -1107,7 +1115,7 @@ def update_setting(self, key, value):
</dd>
<dt id="connpy.grpc_layer.stubs.ExecutionStub"><code class="flex name class">
<span>class <span class="ident">ExecutionStub</span></span>
<span>(</span><span>channel, remote_host)</span>
<span>(</span><span>channel, remote_host, config=None, node_stub=None)</span>
</code></dt>
<dd>
<details class="source">
@@ -1115,13 +1123,58 @@ def update_setting(self, key, value):
<span>Expand source code</span>
</summary>
<pre><code class="python">class ExecutionStub:
def __init__(self, channel, remote_host):
def __init__(self, channel, remote_host, config=None, node_stub=None):
self.stub = connpy_pb2_grpc.ExecutionServiceStub(channel)
self.remote_host = remote_host
self.config = config
self.node_stub = node_stub
def _get_context_service(self):
if not getattr(self.config, &#34;_enforce_context&#34;, True):
return None
if not isinstance(getattr(self.config, &#34;config&#34;, None), dict):
return None
from ..services.context_service import ContextService
return ContextService(self.config)
def _filter_candidates_by_filter_str(self, candidates: list[str], nodes_filter: str) -&gt; list[str]:
if not nodes_filter:
return list(candidates)
cfg_dict = getattr(self.config, &#34;config&#34;, None)
case = cfg_dict.get(&#34;case&#34;, False) if isinstance(cfg_dict, dict) else False
flags = 0 if case else re.IGNORECASE
return [c for c in candidates if re.search(nodes_filter, c, flags)]
def _resolve_remote_batches(self, nodes_filter):
ctx_svc = self._get_context_service()
if ctx_svc is not None and not ctx_svc.is_default_all_context():
if isinstance(nodes_filter, (list, tuple)):
permitted = ctx_svc.filter_node_list(result=list(nodes_filter))
else:
candidates = (
self.node_stub.list_nodes(filter_str=nodes_filter)
if self.node_stub is not None
else []
)
candidates = self._filter_candidates_by_filter_str(candidates, nodes_filter)
permitted = ctx_svc.filter_node_list(result=candidates)
if not permitted:
raise ConnpyError(f&#34;No nodes found matching filter: {nodes_filter}&#34;)
patterns = [f&#34;^{re.escape(n)}$&#34; for n in permitted]
return [
patterns[i : i + MAX_REMOTE_FILTER_NODES]
for i in range(0, len(patterns), MAX_REMOTE_FILTER_NODES)
]
return [[nodes_filter] if isinstance(nodes_filter, str) else list(nodes_filter)]
@handle_errors
def run_commands(self, nodes_filter, commands, variables=None, parallel=10, timeout=10, folder=None, prompt=None, **kwargs):
nodes_list = [nodes_filter] if isinstance(nodes_filter, str) else list(nodes_filter)
batches = self._resolve_remote_batches(nodes_filter)
final_results = {}
on_complete = kwargs.get(&#34;on_node_complete&#34;)
for nodes_list in batches:
req = connpy_pb2.RunRequest(
nodes=nodes_list,
commands=commands,
@@ -1134,9 +1187,6 @@ def update_setting(self, key, value):
if variables is not None:
req.vars.CopyFrom(to_struct(variables))
final_results = {}
on_complete = kwargs.get(&#34;on_node_complete&#34;)
for response in self.stub.run_commands(req):
if on_complete:
on_complete(response.unique_id, response.output, response.status)
@@ -1149,7 +1199,11 @@ def update_setting(self, key, value):
@handle_errors
def test_commands(self, nodes_filter, commands, expected, variables=None, parallel=10, timeout=10, prompt=None, **kwargs):
nodes_list = [nodes_filter] if isinstance(nodes_filter, str) else list(nodes_filter)
batches = self._resolve_remote_batches(nodes_filter)
final_results = {}
on_complete = kwargs.get(&#34;on_node_complete&#34;)
for nodes_list in batches:
req = connpy_pb2.TestRequest(
nodes=nodes_list,
commands=commands,
@@ -1163,9 +1217,6 @@ def update_setting(self, key, value):
if variables is not None:
req.vars.CopyFrom(to_struct(variables))
final_results = {}
on_complete = kwargs.get(&#34;on_node_complete&#34;)
for response in self.stub.test_commands(req):
result_dict = from_struct(response.test_result) if response.HasField(&#34;test_result&#34;) else {}
if on_complete:
@@ -1176,7 +1227,21 @@ def update_setting(self, key, value):
@handle_errors
def run_cli_script(self, nodes_filter, script_path, parallel=10):
req = connpy_pb2.ScriptRequest(param1=nodes_filter, param2=script_path, parallel=parallel)
ctx_svc = self._get_context_service()
effective_filter = nodes_filter
if ctx_svc is not None and not ctx_svc.is_default_all_context():
candidates = (
self.node_stub.list_nodes(filter_str=nodes_filter)
if self.node_stub is not None
else []
)
if isinstance(nodes_filter, str):
candidates = self._filter_candidates_by_filter_str(candidates, nodes_filter)
permitted = ctx_svc.filter_node_list(result=candidates)
if not permitted:
raise ConnpyError(f&#34;No nodes found matching filter: {nodes_filter}&#34;)
effective_filter = &#34;|&#34;.join(f&#34;^{re.escape(n)}$&#34; for n in permitted)
req = connpy_pb2.ScriptRequest(param1=effective_filter, param2=script_path, parallel=parallel)
return from_struct(self.stub.run_cli_script(req).data)</code></pre>
</details>
<div class="desc"></div>
@@ -1192,7 +1257,21 @@ def update_setting(self, key, value):
</summary>
<pre><code class="python">@handle_errors
def run_cli_script(self, nodes_filter, script_path, parallel=10):
req = connpy_pb2.ScriptRequest(param1=nodes_filter, param2=script_path, parallel=parallel)
ctx_svc = self._get_context_service()
effective_filter = nodes_filter
if ctx_svc is not None and not ctx_svc.is_default_all_context():
candidates = (
self.node_stub.list_nodes(filter_str=nodes_filter)
if self.node_stub is not None
else []
)
if isinstance(nodes_filter, str):
candidates = self._filter_candidates_by_filter_str(candidates, nodes_filter)
permitted = ctx_svc.filter_node_list(result=candidates)
if not permitted:
raise ConnpyError(f&#34;No nodes found matching filter: {nodes_filter}&#34;)
effective_filter = &#34;|&#34;.join(f&#34;^{re.escape(n)}$&#34; for n in permitted)
req = connpy_pb2.ScriptRequest(param1=effective_filter, param2=script_path, parallel=parallel)
return from_struct(self.stub.run_cli_script(req).data)</code></pre>
</details>
<div class="desc"></div>
@@ -1207,7 +1286,11 @@ def run_cli_script(self, nodes_filter, script_path, parallel=10):
</summary>
<pre><code class="python">@handle_errors
def run_commands(self, nodes_filter, commands, variables=None, parallel=10, timeout=10, folder=None, prompt=None, **kwargs):
nodes_list = [nodes_filter] if isinstance(nodes_filter, str) else list(nodes_filter)
batches = self._resolve_remote_batches(nodes_filter)
final_results = {}
on_complete = kwargs.get(&#34;on_node_complete&#34;)
for nodes_list in batches:
req = connpy_pb2.RunRequest(
nodes=nodes_list,
commands=commands,
@@ -1220,9 +1303,6 @@ def run_commands(self, nodes_filter, commands, variables=None, parallel=10, time
if variables is not None:
req.vars.CopyFrom(to_struct(variables))
final_results = {}
on_complete = kwargs.get(&#34;on_node_complete&#34;)
for response in self.stub.run_commands(req):
if on_complete:
on_complete(response.unique_id, response.output, response.status)
@@ -1245,7 +1325,11 @@ def run_commands(self, nodes_filter, commands, variables=None, parallel=10, time
</summary>
<pre><code class="python">@handle_errors
def test_commands(self, nodes_filter, commands, expected, variables=None, parallel=10, timeout=10, prompt=None, **kwargs):
nodes_list = [nodes_filter] if isinstance(nodes_filter, str) else list(nodes_filter)
batches = self._resolve_remote_batches(nodes_filter)
final_results = {}
on_complete = kwargs.get(&#34;on_node_complete&#34;)
for nodes_list in batches:
req = connpy_pb2.TestRequest(
nodes=nodes_list,
commands=commands,
@@ -1259,9 +1343,6 @@ def test_commands(self, nodes_filter, commands, expected, variables=None, parall
if variables is not None:
req.vars.CopyFrom(to_struct(variables))
final_results = {}
on_complete = kwargs.get(&#34;on_node_complete&#34;)
for response in self.stub.test_commands(req):
result_dict = from_struct(response.test_result) if response.HasField(&#34;test_result&#34;) else {}
if on_complete:
@@ -1372,8 +1453,48 @@ def set_reserved_names(self, names):
self.remote_host = remote_host
self.config = config
def _get_context_service(self):
if not getattr(self.config, &#34;_enforce_context&#34;, True):
return None
if not isinstance(getattr(self.config, &#34;config&#34;, None), dict):
return None
from ..services.context_service import ContextService
return ContextService(self.config)
def _ensure_in_active_context(
self, unique_id: str, *, is_folder: bool = False, not_found_msg: str | None = None
) -&gt; None:
ctx_svc = self._get_context_service()
if (
ctx_svc is not None
and not ctx_svc.is_default_all_context()
and not ctx_svc.matches_active_context(unique_id)
):
if not_found_msg is not None:
raise NodeNotFoundError(not_found_msg)
label = &#34;Folder&#34; if is_folder else &#34;Node&#34;
raise NodeNotFoundError(f&#34;{label} &#39;{unique_id}&#39; not found.&#34;)
@handle_errors
def _get_raw_nodes(self) -&gt; list[str]:
req = connpy_pb2.FilterRequest(filter_str=&#34;&#34;, format_str=&#34;&#34;)
return from_value(self.stub.list_nodes(req).data) or []
@handle_errors
def _get_raw_folders(self) -&gt; list[str]:
req = connpy_pb2.FilterRequest(filter_str=&#34;&#34;)
return from_value(self.stub.list_folders(req).data) or []
def _handle_remote_copilot(self, res, request_queue, response_queue, client_buffer_bytes, pause_generator, resume_generator, old_tty):
import json, asyncio, termios, sys, tty, queue, os
import asyncio
import json
import os
import sys
import termios
import tty
from ..cli.terminal_ui import CopilotInterface
from ..core import copilot_terminal_mode
from . import connpy_pb2
@@ -1467,6 +1588,7 @@ def set_reserved_names(self, names):
@handle_errors
def connect_node(self, unique_id, sftp=False, debug=False, logger=None):
self._ensure_in_active_context(unique_id, is_folder=False)
import sys
import select
import tty
@@ -1758,16 +1880,26 @@ def set_reserved_names(self, names):
@handle_errors
def list_nodes(self, filter_str=None, format_str=None):
req = connpy_pb2.FilterRequest(filter_str=filter_str or &#34;&#34;, format_str=format_str or &#34;&#34;)
return from_value(self.stub.list_nodes(req).data) or []
nodes = from_value(self.stub.list_nodes(req).data) or []
if not format_str:
ctx_svc = self._get_context_service()
if ctx_svc is not None and not ctx_svc.is_default_all_context():
nodes = ctx_svc.filter_node_list(result=nodes)
return nodes
@MethodHook
@handle_errors
def list_folders(self, filter_str=None):
req = connpy_pb2.FilterRequest(filter_str=filter_str or &#34;&#34;)
return from_value(self.stub.list_folders(req).data) or []
folders = from_value(self.stub.list_folders(req).data) or []
ctx_svc = self._get_context_service()
if ctx_svc is not None and not ctx_svc.is_default_all_context():
folders = ctx_svc.filter_node_list(result=folders)
return folders
@handle_errors
def get_node_details(self, unique_id):
self._ensure_in_active_context(unique_id, is_folder=False)
return from_struct(self.stub.get_node_details(connpy_pb2.IdRequest(id=unique_id)).data)
@handle_errors
@@ -1775,7 +1907,11 @@ def set_reserved_names(self, names):
return from_value(self.stub.explode_unique(connpy_pb2.IdRequest(id=unique_id)).data)
@handle_errors
def validate_parent_folder(self, unique_id):
def validate_parent_folder(self, unique_id, is_folder=False):
ctx_svc = self._get_context_service()
if ctx_svc is not None:
ctx_svc.validate_in_active_context(unique_id, is_folder=is_folder)
if not is_folder:
self.stub.validate_parent_folder(connpy_pb2.IdRequest(id=unique_id))
@handle_errors
@@ -1792,7 +1928,7 @@ def set_reserved_names(self, names):
# node cache is what matters for fzf. We&#39;ll fetch profiles if we can.
# For now, let&#39;s sync what we have.
if nodes is not None or folders is not None or profiles is not None:
if self.config is not None and (nodes is not None or folders is not None or profiles is not None):
self.config._generate_nodes_cache(nodes=nodes, folders=folders, profiles=profiles)
def _trigger_local_cache_sync(self):
@@ -1807,12 +1943,16 @@ def set_reserved_names(self, names):
@handle_errors
def add_node(self, unique_id, data, is_folder=False):
ctx_svc = self._get_context_service()
if ctx_svc is not None:
ctx_svc.validate_in_active_context(unique_id, is_folder=is_folder)
req = connpy_pb2.NodeRequest(id=unique_id, data=to_struct(data), is_folder=is_folder)
self.stub.add_node(req)
self._trigger_local_cache_sync()
@handle_errors
def update_node(self, unique_id, data, save=True):
self._ensure_in_active_context(unique_id, is_folder=False)
req = connpy_pb2.NodeRequest(id=unique_id, data=to_struct(data), is_folder=False)
self.stub.update_node(req)
if save:
@@ -1820,6 +1960,7 @@ def set_reserved_names(self, names):
@handle_errors
def delete_node(self, unique_id, is_folder=False, save=True):
self._ensure_in_active_context(unique_id, is_folder=is_folder)
req = connpy_pb2.DeleteRequest(id=unique_id, is_folder=is_folder)
self.stub.delete_node(req)
if save:
@@ -1827,12 +1968,22 @@ def set_reserved_names(self, names):
@handle_errors
def move_node(self, src_id, dst_id, copy=False):
self._ensure_in_active_context(
src_id, is_folder=False, not_found_msg=f&#34;Source node &#39;{src_id}&#39; not found.&#34;
)
ctx_svc = self._get_context_service()
if ctx_svc is not None:
ctx_svc.validate_in_active_context(dst_id, is_folder=False)
req = connpy_pb2.MoveRequest(src_id=src_id, dst_id=dst_id, copy=copy)
self.stub.move_node(req)
self._trigger_local_cache_sync()
@handle_errors
def bulk_add(self, ids, hosts, common_data):
ctx_svc = self._get_context_service()
if ctx_svc is not None:
for uid in ids:
ctx_svc.validate_in_active_context(uid, is_folder=False)
req = connpy_pb2.BulkRequest(ids=ids, hosts=hosts, common_data=to_struct(common_data))
self.stub.bulk_add(req)
self._trigger_local_cache_sync()
@@ -1872,6 +2023,9 @@ def set_reserved_names(self, names):
</summary>
<pre><code class="python">@handle_errors
def add_node(self, unique_id, data, is_folder=False):
ctx_svc = self._get_context_service()
if ctx_svc is not None:
ctx_svc.validate_in_active_context(unique_id, is_folder=is_folder)
req = connpy_pb2.NodeRequest(id=unique_id, data=to_struct(data), is_folder=is_folder)
self.stub.add_node(req)
self._trigger_local_cache_sync()</code></pre>
@@ -1888,6 +2042,10 @@ def add_node(self, unique_id, data, is_folder=False):
</summary>
<pre><code class="python">@handle_errors
def bulk_add(self, ids, hosts, common_data):
ctx_svc = self._get_context_service()
if ctx_svc is not None:
for uid in ids:
ctx_svc.validate_in_active_context(uid, is_folder=False)
req = connpy_pb2.BulkRequest(ids=ids, hosts=hosts, common_data=to_struct(common_data))
self.stub.bulk_add(req)
self._trigger_local_cache_sync()</code></pre>
@@ -2059,6 +2217,7 @@ def connect_dynamic(self, connection_params, debug=False):
</summary>
<pre><code class="python">@handle_errors
def connect_node(self, unique_id, sftp=False, debug=False, logger=None):
self._ensure_in_active_context(unique_id, is_folder=False)
import sys
import select
import tty
@@ -2213,6 +2372,7 @@ def connect_node(self, unique_id, sftp=False, debug=False, logger=None):
</summary>
<pre><code class="python">@handle_errors
def delete_node(self, unique_id, is_folder=False, save=True):
self._ensure_in_active_context(unique_id, is_folder=is_folder)
req = connpy_pb2.DeleteRequest(id=unique_id, is_folder=is_folder)
self.stub.delete_node(req)
if save:
@@ -2275,7 +2435,7 @@ def generate_cache(self, nodes=None, folders=None, profiles=None):
# node cache is what matters for fzf. We&#39;ll fetch profiles if we can.
# For now, let&#39;s sync what we have.
if nodes is not None or folders is not None or profiles is not None:
if self.config is not None and (nodes is not None or folders is not None or profiles is not None):
self.config._generate_nodes_cache(nodes=nodes, folders=folders, profiles=profiles)</code></pre>
</details>
<div class="desc"></div>
@@ -2308,6 +2468,7 @@ def get_inventory(self):
</summary>
<pre><code class="python">@handle_errors
def get_node_details(self, unique_id):
self._ensure_in_active_context(unique_id, is_folder=False)
return from_struct(self.stub.get_node_details(connpy_pb2.IdRequest(id=unique_id)).data)</code></pre>
</details>
<div class="desc"></div>
@@ -2324,7 +2485,11 @@ def get_node_details(self, unique_id):
@handle_errors
def list_folders(self, filter_str=None):
req = connpy_pb2.FilterRequest(filter_str=filter_str or &#34;&#34;)
return from_value(self.stub.list_folders(req).data) or []</code></pre>
folders = from_value(self.stub.list_folders(req).data) or []
ctx_svc = self._get_context_service()
if ctx_svc is not None and not ctx_svc.is_default_all_context():
folders = ctx_svc.filter_node_list(result=folders)
return folders</code></pre>
</details>
<div class="desc"></div>
</dd>
@@ -2340,7 +2505,12 @@ def list_folders(self, filter_str=None):
@handle_errors
def list_nodes(self, filter_str=None, format_str=None):
req = connpy_pb2.FilterRequest(filter_str=filter_str or &#34;&#34;, format_str=format_str or &#34;&#34;)
return from_value(self.stub.list_nodes(req).data) or []</code></pre>
nodes = from_value(self.stub.list_nodes(req).data) or []
if not format_str:
ctx_svc = self._get_context_service()
if ctx_svc is not None and not ctx_svc.is_default_all_context():
nodes = ctx_svc.filter_node_list(result=nodes)
return nodes</code></pre>
</details>
<div class="desc"></div>
</dd>
@@ -2354,6 +2524,12 @@ def list_nodes(self, filter_str=None, format_str=None):
</summary>
<pre><code class="python">@handle_errors
def move_node(self, src_id, dst_id, copy=False):
self._ensure_in_active_context(
src_id, is_folder=False, not_found_msg=f&#34;Source node &#39;{src_id}&#39; not found.&#34;
)
ctx_svc = self._get_context_service()
if ctx_svc is not None:
ctx_svc.validate_in_active_context(dst_id, is_folder=False)
req = connpy_pb2.MoveRequest(src_id=src_id, dst_id=dst_id, copy=copy)
self.stub.move_node(req)
self._trigger_local_cache_sync()</code></pre>
@@ -2385,6 +2561,7 @@ def set_reserved_names(self, names):
</summary>
<pre><code class="python">@handle_errors
def update_node(self, unique_id, data, save=True):
self._ensure_in_active_context(unique_id, is_folder=False)
req = connpy_pb2.NodeRequest(id=unique_id, data=to_struct(data), is_folder=False)
self.stub.update_node(req)
if save:
@@ -2393,7 +2570,7 @@ def update_node(self, unique_id, data, save=True):
<div class="desc"></div>
</dd>
<dt id="connpy.grpc_layer.stubs.NodeStub.validate_parent_folder"><code class="name flex">
<span>def <span class="ident">validate_parent_folder</span></span>(<span>self, unique_id)</span>
<span>def <span class="ident">validate_parent_folder</span></span>(<span>self, unique_id, is_folder=False)</span>
</code></dt>
<dd>
<details class="source">
@@ -2401,7 +2578,11 @@ def update_node(self, unique_id, data, save=True):
<span>Expand source code</span>
</summary>
<pre><code class="python">@handle_errors
def validate_parent_folder(self, unique_id):
def validate_parent_folder(self, unique_id, is_folder=False):
ctx_svc = self._get_context_service()
if ctx_svc is not None:
ctx_svc.validate_in_active_context(unique_id, is_folder=is_folder)
if not is_folder:
self.stub.validate_parent_folder(connpy_pb2.IdRequest(id=unique_id))</code></pre>
</details>
<div class="desc"></div>
+2 -2
View File
@@ -113,7 +113,7 @@ el.replaceWith(d);
try:
# Attempt a fresh configuration load
config = configfile(conf=conf_file, shared_config=self._shared_config)
new_provider = ServiceProvider(config, mode=&#34;local&#34;)
new_provider = ServiceProvider(config, mode=&#34;local&#34;, enforce_context=False)
# Successfully loaded, clean up the old provider
if old_provider:
@@ -221,7 +221,7 @@ el.replaceWith(d);
try:
# Attempt a fresh configuration load
config = configfile(conf=conf_file, shared_config=self._shared_config)
new_provider = ServiceProvider(config, mode=&#34;local&#34;)
new_provider = ServiceProvider(config, mode=&#34;local&#34;, enforce_context=False)
# Successfully loaded, clean up the old provider
if old_provider:
+3074 -678
View File
File diff suppressed because it is too large. Load diff
+148 -59
View File
@@ -58,29 +58,50 @@ el.replaceWith(d);
<pre><code class="python">class MCPClientManager:
&#34;&#34;&#34;Manages MCP SSE client connections for connpy.&#34;&#34;&#34;
_instance = None
_instances: ClassVar[dict[Any, &#34;MCPClientManager&#34;]] = {}
_instance: ClassVar[&#34;MCPClientManager | None&#34;] = None
_lock = threading.Lock()
def __new__(cls, *args, **kwargs):
@classmethod
def _resolve_scope_key(cls, config=None, *args, **kwargs) -&gt; Any:
cfg = config
if cfg is None and &#34;config&#34; in kwargs:
cfg = kwargs[&#34;config&#34;]
if cfg is not None:
defaultdir = getattr(cfg, &#34;defaultdir&#34;, None)
if defaultdir:
return os.path.abspath(str(defaultdir))
return id(cfg)
return &#34;default&#34;
def __new__(cls, config=None, *args, **kwargs):
key = cls._resolve_scope_key(config, *args, **kwargs)
with cls._lock:
if cls._instance is None:
cls._instance = super(MCPClientManager, cls).__new__(cls)
cls._instance._initialized = False
return cls._instance
if cls._instance is None and cls._instances:
cls._instances.clear()
if key not in cls._instances:
instance = super().__new__(cls)
instance._initialized = False
instance._scope_key = key
cls._instances[key] = instance
cls._instance = cls._instances[key]
return cls._instances[key]
def __init__(self, config=None):
if self._initialized:
return
if config is not None:
self.config = config
self.sessions: Dict[str, Dict[str, Any]] = {} # name -&gt; {session, stack}
self.tool_cache: Dict[str, List[Dict[str, Any]]] = {}
self._connecting: Dict[str, asyncio.Future] = {}
if getattr(self, &#34;_initialized&#34;, False):
return
if not hasattr(self, &#34;config&#34;):
self.config = config
self.sessions: dict[str, dict[str, Any]] = {} # name -&gt; {session, stack}
self.tool_cache: dict[str, list[dict[str, Any]]] = {}
self._server_configs: dict[str, dict[str, Any]] = {}
self._connecting: dict[str, asyncio.Future] = {}
self._initialized = True
async def get_tools_for_llm(self, os_filter: Optional[str] = None) -&gt; List[Dict[str, Any]]:
&#34;&#34;&#34;
Fetches tools from enabled MCP servers that match the OS filter.
&#34;&#34;&#34;
async def get_tools_for_llm(self, os_filter: str | None = None) -&gt; list[dict[str, Any]]:
&#34;&#34;&#34;Fetches tools from enabled MCP servers that match the OS filter.&#34;&#34;&#34;
if not MCP_AVAILABLE:
return []
@@ -90,29 +111,53 @@ el.replaceWith(d);
mcp_config = self.config.get_effective_setting(&#34;ai&#34;, {}).get(&#34;mcp_servers&#34;, {})
else:
mcp_config = self.config.config.get(&#34;ai&#34;, {}).get(&#34;mcp_servers&#34;, {}) if hasattr(self.config, &#34;config&#34;) else {}
except Exception:
except Exception: # noqa: BLE001
return []
if not isinstance(mcp_config, dict):
mcp_config = {}
# Invalidate cache and reconnect if server configuration changed or server was removed
current_names = set(mcp_config.keys())
for old_name in list(self._server_configs.keys()):
if old_name not in current_names:
self.tool_cache.pop(old_name, None)
old_data = self.sessions.pop(old_name, None)
if old_data and old_data.get(&#34;stack&#34;):
with suppress(Exception):
await old_data[&#34;stack&#34;].aclose()
self._server_configs.pop(old_name, None)
for name, cfg in mcp_config.items():
cached_cfg = self._server_configs.get(name)
if cached_cfg is not None and cached_cfg != cfg:
self.tool_cache.pop(name, None)
old_data = self.sessions.pop(name, None)
if old_data and old_data.get(&#34;stack&#34;):
with suppress(Exception):
await old_data[&#34;stack&#34;].aclose()
self._server_configs[name] = dict(cfg) if isinstance(cfg, dict) else cfg
async def _fetch(name, cfg):
if not cfg.get(&#34;enabled&#34;, True): return []
if not isinstance(cfg, dict) or not cfg.get(&#34;enabled&#34;, True):
return []
# Filter by OS if specified in config (primarily used for copilot strict matching)
auto_os = cfg.get(&#34;auto_load_on_os&#34;)
if os_filter is not None and auto_os and os_filter.lower() != auto_os.lower():
return []
try:
with suppress(Exception):
session = await self._ensure_connected(name, cfg)
if session:
if name in self.tool_cache: return self.tool_cache[name]
if name in self.tool_cache:
return self.tool_cache[name]
llm_tools = await self._fetch_tools_as_openai(name, session)
self.tool_cache[name] = llm_tools
return llm_tools
except Exception:
pass
return []
tasks = [ _fetch(name, cfg) for name, cfg in mcp_config.items() ]
tasks = [_fetch(name, cfg) for name, cfg in mcp_config.items()]
if tasks:
results = await asyncio.gather(*tasks)
@@ -121,8 +166,9 @@ el.replaceWith(d);
return all_llm_tools
async def _ensure_connected(self, name: str, cfg: Dict[str, Any]) -&gt; Optional[Any]:
if not MCP_AVAILABLE: return None
async def _ensure_connected(self, name: str, cfg: dict[str, Any]) -&gt; Any | None:
if not MCP_AVAILABLE:
return None
if name in self.sessions and self.sessions[name].get(&#34;session&#34;):
return self.sessions[name][&#34;session&#34;]
@@ -134,7 +180,7 @@ el.replaceWith(d);
if name in self._connecting:
try:
return await asyncio.wait_for(asyncio.shield(self._connecting[name]), timeout=10.0)
except Exception:
except Exception: # noqa: BLE001
return None
loop = asyncio.get_running_loop()
@@ -155,33 +201,32 @@ el.replaceWith(d);
self.sessions[name] = {&#34;session&#34;: session, &#34;stack&#34;: stack}
fut.set_result(session)
return session
except Exception:
except Exception: # noqa: BLE001
fut.set_result(None)
return None
finally:
if name in self._connecting:
del self._connecting[name]
async def _fetch_tools_as_openai(self, server_name: str, session: Any) -&gt; List[Dict[str, Any]]:
async def _fetch_tools_as_openai(self, server_name: str, session: Any) -&gt; list[dict[str, Any]]:
try:
result = await asyncio.wait_for(session.list_tools(), timeout=5.0)
openai_tools = []
for tool in result.tools:
# Use mcp_ prefix to ensure valid function name for LiteLLM/Gemini
prefixed_name = f&#34;mcp_{server_name}__{tool.name}&#34;
openai_tools.append({
&#34;type&#34;: &#34;function&#34;,
&#34;function&#34;: {
&#34;name&#34;: prefixed_name,
&#34;description&#34;: f&#34;[{server_name}] {tool.description}&#34;,
&#34;parameters&#34;: tool.inputSchema
}
&#34;parameters&#34;: tool.inputSchema,
},
})
return openai_tools
except Exception:
except Exception: # noqa: BLE001
return []
async def call_tool(self, full_tool_name: str, arguments: Dict[str, Any]) -&gt; Any:
async def call_tool(self, full_tool_name: str, arguments: dict[str, Any]) -&gt; Any:
&#34;&#34;&#34;Calls an MCP tool and returns text result.&#34;&#34;&#34;
if not MCP_AVAILABLE:
return &#34;Error: MCP SDK is not installed.&#34;
@@ -189,7 +234,7 @@ el.replaceWith(d);
if &#34;__&#34; not in full_tool_name:
return f&#34;Error: Tool {full_tool_name} is not a valid MCP tool.&#34;
clean_name = full_tool_name[4:] if full_tool_name.startswith(&#34;mcp_&#34;) else full_tool_name
clean_name = full_tool_name.removeprefix(&#34;mcp_&#34;)
server_name, tool_name = clean_name.split(&#34;__&#34;, 1)
if server_name not in self.sessions:
@@ -200,29 +245,40 @@ el.replaceWith(d);
result = await asyncio.wait_for(session.call_tool(tool_name, arguments), timeout=60.0)
text_outputs = [content.text for content in result.content if hasattr(content, &#34;text&#34;)]
return &#34;\n&#34;.join(text_outputs) if text_outputs else str(result)
except Exception as e:
return f&#34;Error calling tool {tool_name} on {server_name}: {str(e)}&#34;
except Exception as e: # noqa: BLE001
return f&#34;Error calling tool {tool_name} on {server_name}: {e!s}&#34;
async def shutdown(self):
&#34;&#34;&#34;Close all SSE connections.&#34;&#34;&#34;
for name, data in self.sessions.items():
&#34;&#34;&#34;Close all SSE connections and clean up resources.&#34;&#34;&#34;
for data in list(self.sessions.values()):
stack = data.get(&#34;stack&#34;)
if stack:
with suppress(Exception):
await stack.aclose()
self.sessions = {}</code></pre>
self.sessions.clear()
self.tool_cache.clear()
self._server_configs.clear()
for fut in list(self._connecting.values()):
if not fut.done():
fut.cancel()
self._connecting.clear()
with self._lock:
if hasattr(self, &#34;_scope_key&#34;) and self._scope_key in self._instances:
del self._instances[self._scope_key]
self._initialized = False</code></pre>
</details>
<div class="desc"><p>Manages MCP SSE client connections for connpy.</p></div>
<h3>Methods</h3>
<dl>
<dt id="connpy.mcp_client.MCPClientManager.call_tool"><code class="name flex">
<span>async def <span class="ident">call_tool</span></span>(<span>self, full_tool_name: str, arguments: Dict[str, Any]) ‑> Any</span>
<span>async def <span class="ident">call_tool</span></span>(<span>self, full_tool_name: str, arguments: dict[str, typing.Any]) ‑> Any</span>
</code></dt>
<dd>
<details class="source">
<summary>
<span>Expand source code</span>
</summary>
<pre><code class="python">async def call_tool(self, full_tool_name: str, arguments: Dict[str, Any]) -&gt; Any:
<pre><code class="python">async def call_tool(self, full_tool_name: str, arguments: dict[str, Any]) -&gt; Any:
&#34;&#34;&#34;Calls an MCP tool and returns text result.&#34;&#34;&#34;
if not MCP_AVAILABLE:
return &#34;Error: MCP SDK is not installed.&#34;
@@ -230,7 +286,7 @@ el.replaceWith(d);
if &#34;__&#34; not in full_tool_name:
return f&#34;Error: Tool {full_tool_name} is not a valid MCP tool.&#34;
clean_name = full_tool_name[4:] if full_tool_name.startswith(&#34;mcp_&#34;) else full_tool_name
clean_name = full_tool_name.removeprefix(&#34;mcp_&#34;)
server_name, tool_name = clean_name.split(&#34;__&#34;, 1)
if server_name not in self.sessions:
@@ -241,23 +297,21 @@ el.replaceWith(d);
result = await asyncio.wait_for(session.call_tool(tool_name, arguments), timeout=60.0)
text_outputs = [content.text for content in result.content if hasattr(content, &#34;text&#34;)]
return &#34;\n&#34;.join(text_outputs) if text_outputs else str(result)
except Exception as e:
return f&#34;Error calling tool {tool_name} on {server_name}: {str(e)}&#34;</code></pre>
except Exception as e: # noqa: BLE001
return f&#34;Error calling tool {tool_name} on {server_name}: {e!s}&#34;</code></pre>
</details>
<div class="desc"><p>Calls an MCP tool and returns text result.</p></div>
</dd>
<dt id="connpy.mcp_client.MCPClientManager.get_tools_for_llm"><code class="name flex">
<span>async def <span class="ident">get_tools_for_llm</span></span>(<span>self, os_filter: str | None = None) ‑> List[Dict[str, Any]]</span>
<span>async def <span class="ident">get_tools_for_llm</span></span>(<span>self, os_filter: str | None = None) ‑> list[dict[str, typing.Any]]</span>
</code></dt>
<dd>
<details class="source">
<summary>
<span>Expand source code</span>
</summary>
<pre><code class="python">async def get_tools_for_llm(self, os_filter: Optional[str] = None) -&gt; List[Dict[str, Any]]:
&#34;&#34;&#34;
Fetches tools from enabled MCP servers that match the OS filter.
&#34;&#34;&#34;
<pre><code class="python">async def get_tools_for_llm(self, os_filter: str | None = None) -&gt; list[dict[str, Any]]:
&#34;&#34;&#34;Fetches tools from enabled MCP servers that match the OS filter.&#34;&#34;&#34;
if not MCP_AVAILABLE:
return []
@@ -267,29 +321,53 @@ el.replaceWith(d);
mcp_config = self.config.get_effective_setting(&#34;ai&#34;, {}).get(&#34;mcp_servers&#34;, {})
else:
mcp_config = self.config.config.get(&#34;ai&#34;, {}).get(&#34;mcp_servers&#34;, {}) if hasattr(self.config, &#34;config&#34;) else {}
except Exception:
except Exception: # noqa: BLE001
return []
if not isinstance(mcp_config, dict):
mcp_config = {}
# Invalidate cache and reconnect if server configuration changed or server was removed
current_names = set(mcp_config.keys())
for old_name in list(self._server_configs.keys()):
if old_name not in current_names:
self.tool_cache.pop(old_name, None)
old_data = self.sessions.pop(old_name, None)
if old_data and old_data.get(&#34;stack&#34;):
with suppress(Exception):
await old_data[&#34;stack&#34;].aclose()
self._server_configs.pop(old_name, None)
for name, cfg in mcp_config.items():
cached_cfg = self._server_configs.get(name)
if cached_cfg is not None and cached_cfg != cfg:
self.tool_cache.pop(name, None)
old_data = self.sessions.pop(name, None)
if old_data and old_data.get(&#34;stack&#34;):
with suppress(Exception):
await old_data[&#34;stack&#34;].aclose()
self._server_configs[name] = dict(cfg) if isinstance(cfg, dict) else cfg
async def _fetch(name, cfg):
if not cfg.get(&#34;enabled&#34;, True): return []
if not isinstance(cfg, dict) or not cfg.get(&#34;enabled&#34;, True):
return []
# Filter by OS if specified in config (primarily used for copilot strict matching)
auto_os = cfg.get(&#34;auto_load_on_os&#34;)
if os_filter is not None and auto_os and os_filter.lower() != auto_os.lower():
return []
try:
with suppress(Exception):
session = await self._ensure_connected(name, cfg)
if session:
if name in self.tool_cache: return self.tool_cache[name]
if name in self.tool_cache:
return self.tool_cache[name]
llm_tools = await self._fetch_tools_as_openai(name, session)
self.tool_cache[name] = llm_tools
return llm_tools
except Exception:
pass
return []
tasks = [ _fetch(name, cfg) for name, cfg in mcp_config.items() ]
tasks = [_fetch(name, cfg) for name, cfg in mcp_config.items()]
if tasks:
results = await asyncio.gather(*tasks)
@@ -309,14 +387,25 @@ el.replaceWith(d);
<span>Expand source code</span>
</summary>
<pre><code class="python">async def shutdown(self):
&#34;&#34;&#34;Close all SSE connections.&#34;&#34;&#34;
for name, data in self.sessions.items():
&#34;&#34;&#34;Close all SSE connections and clean up resources.&#34;&#34;&#34;
for data in list(self.sessions.values()):
stack = data.get(&#34;stack&#34;)
if stack:
with suppress(Exception):
await stack.aclose()
self.sessions = {}</code></pre>
self.sessions.clear()
self.tool_cache.clear()
self._server_configs.clear()
for fut in list(self._connecting.values()):
if not fut.done():
fut.cancel()
self._connecting.clear()
with self._lock:
if hasattr(self, &#34;_scope_key&#34;) and self._scope_key in self._instances:
del self._instances[self._scope_key]
self._initialized = False</code></pre>
</details>
<div class="desc"><p>Close all SSE connections.</p></div>
<div class="desc"><p>Close all SSE connections and clean up resources.</p></div>
</dd>
</dl>
</dd>
File diff suppressed because it is too large. Load diff
+2 -1
View File
@@ -427,8 +427,9 @@ el.replaceWith(d);
</details>
<div class="desc"><p>Business logic for interacting with AI agents and LLM configurations.</p>
<p>Initialize the service.</p>
<h2 id="args">Args</h2>
<dl>
<dt>Args</dt>
<dt>-----=</dt>
<dt><strong><code>config</code></strong></dt>
<dd>An instance of configfile (or None to instantiate a new one/use global context).</dd>
</dl></div>
+2 -1
View File
@@ -89,8 +89,9 @@ el.replaceWith(d);
</details>
<div class="desc"><p>Base class for all connpy services, providing common configuration access.</p>
<p>Initialize the service.</p>
<h2 id="args">Args</h2>
<dl>
<dt>Args</dt>
<dt>-----=</dt>
<dt><strong><code>config</code></strong></dt>
<dd>An instance of configfile (or None to instantiate a new one/use global context).</dd>
</dl></div>
+2 -1
View File
@@ -133,8 +133,9 @@ el.replaceWith(d);
</details>
<div class="desc"><p>Business logic for general application settings and state configuration.</p>
<p>Initialize the service.</p>
<h2 id="args">Args</h2>
<dl>
<dt>Args</dt>
<dt>-----=</dt>
<dt><strong><code>config</code></strong></dt>
<dd>An instance of configfile (or None to instantiate a new one/use global context).</dd>
</dl></div>
+146 -28
View File
@@ -58,15 +58,31 @@ el.replaceWith(d);
<pre><code class="python">class ContextService(BaseService):
&#34;&#34;&#34;Business logic for managing and applying regex-based contexts locally.&#34;&#34;&#34;
@staticmethod
def _validate_regexes(regexes: Any) -&gt; None:
if not isinstance(regexes, list) or not (1 &lt;= len(regexes) &lt;= MAX_CONTEXT_REGEXES):
raise ValueError(
f&#34;Context regexes must be a list of 1 to {MAX_CONTEXT_REGEXES} patterns&#34;
)
for pat in regexes:
if not isinstance(pat, str) or not (1 &lt;= len(pat) &lt;= MAX_CONTEXT_PATTERN_LEN):
raise ValueError(
f&#34;Each context regex must be a non-empty string up to {MAX_CONTEXT_PATTERN_LEN} characters&#34;
)
try:
re.compile(pat)
except re.error as exc:
raise ValueError(f&#34;Invalid context regex {pat!r}: {exc}&#34;) from None
@property
def contexts(self) -&gt; Dict[str, List[str]]:
def contexts(self) -&gt; dict[str, list[str]]:
return self.config.config.get(&#34;contexts&#34;, {&#34;all&#34;: [&#34;.*&#34;]})
@property
def current_context(self) -&gt; str:
return self.config.config.get(&#34;current_context&#34;, &#34;all&#34;)
def list_contexts(self) -&gt; List[Dict[str, Any]]:
def list_contexts(self) -&gt; list[dict[str, Any]]:
result = []
for name in self.contexts.keys():
result.append({
@@ -76,27 +92,29 @@ el.replaceWith(d);
})
return result
def add_context(self, name: str, regexes: List[str]):
def add_context(self, name: str, regexes: list[str]):
if not name.isalnum():
raise ValueError(&#34;Context name must be alphanumeric&#34;)
self._validate_regexes(regexes)
ctxs = self.contexts
if name in ctxs:
raise ValueError(f&#34;Context &#39;{name}&#39; already exists&#34;)
ctxs[name] = regexes
ctxs[name] = list(regexes)
self.config.config[&#34;contexts&#34;] = ctxs
self.config._saveconfig(self.config.file)
def update_context(self, name: str, regexes: List[str]):
def update_context(self, name: str, regexes: list[str]):
if name == &#34;all&#34;:
raise ValueError(&#34;Cannot modify default context &#39;all&#39;&#34;)
self._validate_regexes(regexes)
ctxs = self.contexts
if name not in ctxs:
raise ValueError(f&#34;Context &#39;{name}&#39; does not exist&#34;)
ctxs[name] = regexes
ctxs[name] = list(regexes)
self.config.config[&#34;contexts&#34;] = ctxs
self.config._saveconfig(self.config.file)
@@ -118,15 +136,47 @@ el.replaceWith(d);
if name not in self.contexts:
raise ValueError(f&#34;Context &#39;{name}&#39; does not exist&#34;)
prev_context = self.current_context
self.config.config[&#34;current_context&#34;] = name
self.config._saveconfig(self.config.file)
can_refresh = (
self.config._is_context_filtering_enabled_unlocked()
if (
not hasattr(self.config, &#34;assert_called&#34;)
and callable(getattr(self.config, &#34;_is_context_filtering_enabled_unlocked&#34;, None))
)
else getattr(self.config, &#34;_enforce_context&#34;, True)
)
if (
name == &#34;all&#34;
and prev_context != &#34;all&#34;
and can_refresh
and hasattr(self.config, &#34;_generate_nodes_cache&#34;)
):
self.config._generate_nodes_cache()
def get_active_regexes(self) -&gt; List[re.Pattern]:
patterns = self.contexts.get(self.current_context, [&#34;.*&#34;])
return [re.compile(p) for p in patterns]
def get_active_regexes(self) -&gt; list[re.Pattern]:
return compile_active_context_patterns(getattr(self.config, &#34;config&#34;, None))
def _match_any(self, node_name: str, patterns: List[re.Pattern]) -&gt; bool:
return any(p.match(node_name) for p in patterns)
def _match_any(self, node_name: str, patterns: list[re.Pattern]) -&gt; bool:
return match_active_context_patterns(node_name, patterns)
def is_default_all_context(self) -&gt; bool:
return is_default_all_context_cfg(getattr(self.config, &#34;config&#34;, None))
def matches_active_context(self, name: str) -&gt; bool:
if self.is_default_all_context():
return isinstance(name, str) and bool(name)
if not isinstance(name, str) or not name:
return False
return match_active_context_patterns(name, self.get_active_regexes())
def validate_in_active_context(self, unique_id: str, is_folder: bool = False) -&gt; None:
if not self.is_default_all_context() and not self.matches_active_context(unique_id):
label = &#34;Folder&#34; if is_folder else &#34;Node&#34;
raise InvalidConfigurationError(
f&#34;{label} &#39;{unique_id}&#39; does not match active context &#39;{self.current_context}&#39;.&#34;
)
# Hook handlers for filtering
def filter_node_list(self, *args, **kwargs):
@@ -139,8 +189,9 @@ el.replaceWith(d);
</details>
<div class="desc"><p>Business logic for managing and applying regex-based contexts locally.</p>
<p>Initialize the service.</p>
<h2 id="args">Args</h2>
<dl>
<dt>Args</dt>
<dt>-----=</dt>
<dt><strong><code>config</code></strong></dt>
<dd>An instance of configfile (or None to instantiate a new one/use global context).</dd>
</dl></div>
@@ -150,14 +201,14 @@ el.replaceWith(d);
</ul>
<h3>Instance variables</h3>
<dl>
<dt id="connpy.services.context_service.ContextService.contexts"><code class="name">prop <span class="ident">contexts</span> : Dict[str, List[str]]</code></dt>
<dt id="connpy.services.context_service.ContextService.contexts"><code class="name">prop <span class="ident">contexts</span> : dict[str, list[str]]</code></dt>
<dd>
<details class="source">
<summary>
<span>Expand source code</span>
</summary>
<pre><code class="python">@property
def contexts(self) -&gt; Dict[str, List[str]]:
def contexts(self) -&gt; dict[str, list[str]]:
return self.config.config.get(&#34;contexts&#34;, {&#34;all&#34;: [&#34;.*&#34;]})</code></pre>
</details>
<div class="desc"></div>
@@ -178,22 +229,23 @@ def current_context(self) -&gt; str:
<h3>Methods</h3>
<dl>
<dt id="connpy.services.context_service.ContextService.add_context"><code class="name flex">
<span>def <span class="ident">add_context</span></span>(<span>self, name: str, regexes: List[str])</span>
<span>def <span class="ident">add_context</span></span>(<span>self, name: str, regexes: list[str])</span>
</code></dt>
<dd>
<details class="source">
<summary>
<span>Expand source code</span>
</summary>
<pre><code class="python">def add_context(self, name: str, regexes: List[str]):
<pre><code class="python">def add_context(self, name: str, regexes: list[str]):
if not name.isalnum():
raise ValueError(&#34;Context name must be alphanumeric&#34;)
self._validate_regexes(regexes)
ctxs = self.contexts
if name in ctxs:
raise ValueError(f&#34;Context &#39;{name}&#39; already exists&#34;)
ctxs[name] = regexes
ctxs[name] = list(regexes)
self.config.config[&#34;contexts&#34;] = ctxs
self.config._saveconfig(self.config.file)</code></pre>
</details>
@@ -252,28 +304,40 @@ def current_context(self) -&gt; str:
<div class="desc"></div>
</dd>
<dt id="connpy.services.context_service.ContextService.get_active_regexes"><code class="name flex">
<span>def <span class="ident">get_active_regexes</span></span>(<span>self) ‑> List[re.Pattern]</span>
<span>def <span class="ident">get_active_regexes</span></span>(<span>self) ‑> list[re.Pattern]</span>
</code></dt>
<dd>
<details class="source">
<summary>
<span>Expand source code</span>
</summary>
<pre><code class="python">def get_active_regexes(self) -&gt; List[re.Pattern]:
patterns = self.contexts.get(self.current_context, [&#34;.*&#34;])
return [re.compile(p) for p in patterns]</code></pre>
<pre><code class="python">def get_active_regexes(self) -&gt; list[re.Pattern]:
return compile_active_context_patterns(getattr(self.config, &#34;config&#34;, None))</code></pre>
</details>
<div class="desc"></div>
</dd>
<dt id="connpy.services.context_service.ContextService.is_default_all_context"><code class="name flex">
<span>def <span class="ident">is_default_all_context</span></span>(<span>self) ‑> bool</span>
</code></dt>
<dd>
<details class="source">
<summary>
<span>Expand source code</span>
</summary>
<pre><code class="python">def is_default_all_context(self) -&gt; bool:
return is_default_all_context_cfg(getattr(self.config, &#34;config&#34;, None))</code></pre>
</details>
<div class="desc"></div>
</dd>
<dt id="connpy.services.context_service.ContextService.list_contexts"><code class="name flex">
<span>def <span class="ident">list_contexts</span></span>(<span>self) ‑> List[Dict[str, Any]]</span>
<span>def <span class="ident">list_contexts</span></span>(<span>self) ‑> list[dict[str, typing.Any]]</span>
</code></dt>
<dd>
<details class="source">
<summary>
<span>Expand source code</span>
</summary>
<pre><code class="python">def list_contexts(self) -&gt; List[Dict[str, Any]]:
<pre><code class="python">def list_contexts(self) -&gt; list[dict[str, Any]]:
result = []
for name in self.contexts.keys():
result.append({
@@ -285,6 +349,23 @@ def current_context(self) -&gt; str:
</details>
<div class="desc"></div>
</dd>
<dt id="connpy.services.context_service.ContextService.matches_active_context"><code class="name flex">
<span>def <span class="ident">matches_active_context</span></span>(<span>self, name: str) ‑> bool</span>
</code></dt>
<dd>
<details class="source">
<summary>
<span>Expand source code</span>
</summary>
<pre><code class="python">def matches_active_context(self, name: str) -&gt; bool:
if self.is_default_all_context():
return isinstance(name, str) and bool(name)
if not isinstance(name, str) or not name:
return False
return match_active_context_patterns(name, self.get_active_regexes())</code></pre>
</details>
<div class="desc"></div>
</dd>
<dt id="connpy.services.context_service.ContextService.set_active_context"><code class="name flex">
<span>def <span class="ident">set_active_context</span></span>(<span>self, name: str)</span>
</code></dt>
@@ -297,33 +378,67 @@ def current_context(self) -&gt; str:
if name not in self.contexts:
raise ValueError(f&#34;Context &#39;{name}&#39; does not exist&#34;)
prev_context = self.current_context
self.config.config[&#34;current_context&#34;] = name
self.config._saveconfig(self.config.file)</code></pre>
self.config._saveconfig(self.config.file)
can_refresh = (
self.config._is_context_filtering_enabled_unlocked()
if (
not hasattr(self.config, &#34;assert_called&#34;)
and callable(getattr(self.config, &#34;_is_context_filtering_enabled_unlocked&#34;, None))
)
else getattr(self.config, &#34;_enforce_context&#34;, True)
)
if (
name == &#34;all&#34;
and prev_context != &#34;all&#34;
and can_refresh
and hasattr(self.config, &#34;_generate_nodes_cache&#34;)
):
self.config._generate_nodes_cache()</code></pre>
</details>
<div class="desc"></div>
</dd>
<dt id="connpy.services.context_service.ContextService.update_context"><code class="name flex">
<span>def <span class="ident">update_context</span></span>(<span>self, name: str, regexes: List[str])</span>
<span>def <span class="ident">update_context</span></span>(<span>self, name: str, regexes: list[str])</span>
</code></dt>
<dd>
<details class="source">
<summary>
<span>Expand source code</span>
</summary>
<pre><code class="python">def update_context(self, name: str, regexes: List[str]):
<pre><code class="python">def update_context(self, name: str, regexes: list[str]):
if name == &#34;all&#34;:
raise ValueError(&#34;Cannot modify default context &#39;all&#39;&#34;)
self._validate_regexes(regexes)
ctxs = self.contexts
if name not in ctxs:
raise ValueError(f&#34;Context &#39;{name}&#39; does not exist&#34;)
ctxs[name] = regexes
ctxs[name] = list(regexes)
self.config.config[&#34;contexts&#34;] = ctxs
self.config._saveconfig(self.config.file)</code></pre>
</details>
<div class="desc"></div>
</dd>
<dt id="connpy.services.context_service.ContextService.validate_in_active_context"><code class="name flex">
<span>def <span class="ident">validate_in_active_context</span></span>(<span>self, unique_id: str, is_folder: bool = False) ‑> None</span>
</code></dt>
<dd>
<details class="source">
<summary>
<span>Expand source code</span>
</summary>
<pre><code class="python">def validate_in_active_context(self, unique_id: str, is_folder: bool = False) -&gt; None:
if not self.is_default_all_context() and not self.matches_active_context(unique_id):
label = &#34;Folder&#34; if is_folder else &#34;Node&#34;
raise InvalidConfigurationError(
f&#34;{label} &#39;{unique_id}&#39; does not match active context &#39;{self.current_context}&#39;.&#34;
)</code></pre>
</details>
<div class="desc"></div>
</dd>
</dl>
<h3>Inherited members</h3>
<ul class="hlist">
@@ -351,7 +466,7 @@ def current_context(self) -&gt; str:
<ul>
<li>
<h4><code><a title="connpy.services.context_service.ContextService" href="#connpy.services.context_service.ContextService">ContextService</a></code></h4>
<ul class="two-column">
<ul class="">
<li><code><a title="connpy.services.context_service.ContextService.add_context" href="#connpy.services.context_service.ContextService.add_context">add_context</a></code></li>
<li><code><a title="connpy.services.context_service.ContextService.contexts" href="#connpy.services.context_service.ContextService.contexts">contexts</a></code></li>
<li><code><a title="connpy.services.context_service.ContextService.current_context" href="#connpy.services.context_service.ContextService.current_context">current_context</a></code></li>
@@ -359,9 +474,12 @@ def current_context(self) -&gt; str:
<li><code><a title="connpy.services.context_service.ContextService.filter_node_dict" href="#connpy.services.context_service.ContextService.filter_node_dict">filter_node_dict</a></code></li>
<li><code><a title="connpy.services.context_service.ContextService.filter_node_list" href="#connpy.services.context_service.ContextService.filter_node_list">filter_node_list</a></code></li>
<li><code><a title="connpy.services.context_service.ContextService.get_active_regexes" href="#connpy.services.context_service.ContextService.get_active_regexes">get_active_regexes</a></code></li>
<li><code><a title="connpy.services.context_service.ContextService.is_default_all_context" href="#connpy.services.context_service.ContextService.is_default_all_context">is_default_all_context</a></code></li>
<li><code><a title="connpy.services.context_service.ContextService.list_contexts" href="#connpy.services.context_service.ContextService.list_contexts">list_contexts</a></code></li>
<li><code><a title="connpy.services.context_service.ContextService.matches_active_context" href="#connpy.services.context_service.ContextService.matches_active_context">matches_active_context</a></code></li>
<li><code><a title="connpy.services.context_service.ContextService.set_active_context" href="#connpy.services.context_service.ContextService.set_active_context">set_active_context</a></code></li>
<li><code><a title="connpy.services.context_service.ContextService.update_context" href="#connpy.services.context_service.ContextService.update_context">update_context</a></code></li>
<li><code><a title="connpy.services.context_service.ContextService.validate_in_active_context" href="#connpy.services.context_service.ContextService.validate_in_active_context">validate_in_active_context</a></code></li>
</ul>
</li>
</ul>
+1
View File
@@ -67,6 +67,7 @@ el.replaceWith(d);
</ul>
<h3>Subclasses</h3>
<ul class="hlist">
<li><a title="connpy.auth_utils.SanitizedAuthError" href="../auth_utils.html#connpy.auth_utils.SanitizedAuthError">SanitizedAuthError</a></li>
<li><a title="connpy.services.exceptions.ExecutionError" href="#connpy.services.exceptions.ExecutionError">ExecutionError</a></li>
<li><a title="connpy.services.exceptions.InvalidConfigurationError" href="#connpy.services.exceptions.InvalidConfigurationError">InvalidConfigurationError</a></li>
<li><a title="connpy.services.exceptions.NodeAlreadyExistsError" href="#connpy.services.exceptions.NodeAlreadyExistsError">NodeAlreadyExistsError</a></li>
+448 -111
View File
@@ -48,7 +48,7 @@ el.replaceWith(d);
<dl>
<dt id="connpy.services.execution_service.ExecutionService"><code class="flex name class">
<span>class <span class="ident">ExecutionService</span></span>
<span>(</span><span>config=None)</span>
<span>(</span><span>config=None, remote_stub=None, attempt_tracker=None, enforce_context: bool = True)</span>
</code></dt>
<dd>
<details class="source">
@@ -58,23 +58,196 @@ el.replaceWith(d);
<pre><code class="python">class ExecutionService(BaseService):
&#34;&#34;&#34;Business logic for executing commands on nodes and running automation scripts.&#34;&#34;&#34;
def __init__(
self,
config=None,
remote_stub=None,
attempt_tracker=None,
enforce_context: bool = True,
):
super().__init__(config)
self.remote_stub = remote_stub
self.attempt_tracker = (
attempt_tracker if attempt_tracker is not None else AuthAttemptTracker()
)
self.enforce_context = bool(enforce_context)
self.last_fallback_used: dict[str, bool] = {}
def _get_context_service(self):
if (
not getattr(self, &#34;enforce_context&#34;, True)
or not getattr(self.config, &#34;_enforce_context&#34;, True)
or (len(sys.argv) &gt; 1 and sys.argv[1] == &#34;api&#34;)
or not isinstance(getattr(self.config, &#34;config&#34;, None), dict)
):
return None
from .context_service import ContextService
return ContextService(self.config)
def _is_remote_backend(self) -&gt; bool:
&#34;&#34;&#34;Return True when execution is configured to run against a remote gRPC backend.&#34;&#34;&#34;
if getattr(self, &#34;remote_stub&#34;, None) is not None:
return True
cfg = getattr(self.config, &#34;config&#34;, None)
return isinstance(cfg, dict) and cfg.get(&#34;service_mode&#34;) == &#34;remote&#34;
def _ensure_backend_supports_stage_timeouts(
self,
connect_timeout: int | None = None,
auth_timeout: int | None = None,
) -&gt; None:
&#34;&#34;&#34;Raise ConnpyError if per-stage timeout overrides are used on gRPC remote backend.&#34;&#34;&#34;
if self._is_remote_backend() and (connect_timeout is not None or auth_timeout is not None):
raise ConnpyError(
&#34;connect_timeout and auth_timeout overrides are not supported in remote gRPC mode&#34;
)
def _get_remote_channel(self):
cfg = getattr(self.config, &#34;config&#34;, None)
remote_host = cfg.get(&#34;remote_host&#34;) if isinstance(cfg, dict) else None
if not remote_host:
raise ConnpyError(&#34;Remote host must be specified in remote mode&#34;)
import grpc
from connpy.grpc_layer.stubs import AuthClientInterceptor
def get_token():
env_token = os.environ.get(&#34;CONNPY_TOKEN&#34;)
if env_token:
return env_token
defaultdir = getattr(self.config, &#34;defaultdir&#34;, None)
if defaultdir:
token_path = os.path.join(defaultdir, &#34;.token&#34;)
if os.path.exists(token_path):
with suppress(OSError), open(token_path) as f:
return f.read().strip()
return None
channel = grpc.insecure_channel(remote_host)
channel = grpc.intercept_channel(channel, AuthClientInterceptor(get_token))
return channel, remote_host
def _get_remote_stub(self):
&#34;&#34;&#34;Return the configured or lazily constructed ExecutionStub for remote gRPC mode.&#34;&#34;&#34;
if getattr(self, &#34;remote_stub&#34;, None) is not None:
return self.remote_stub
from connpy.grpc_layer.stubs import ExecutionStub
channel, remote_host = self._get_remote_channel()
return ExecutionStub(channel, remote_host=remote_host)
def _filter_remote_nodes_filter(self, nodes_filter):
ctx_svc = self._get_context_service()
if ctx_svc is None or ctx_svc.is_default_all_context():
return nodes_filter
if isinstance(nodes_filter, (list, tuple)):
permitted = ctx_svc.filter_node_list(result=list(nodes_filter))
else:
if self.remote_stub is not None and callable(
getattr(self.remote_stub, &#34;list_nodes&#34;, None)
):
candidates = self.remote_stub.list_nodes(filter_str=nodes_filter)
elif self.remote_stub is not None and callable(
getattr(getattr(self.remote_stub, &#34;node_stub&#34;, None), &#34;list_nodes&#34;, None)
):
candidates = self.remote_stub.node_stub.list_nodes(filter_str=nodes_filter)
elif self.remote_stub is None:
from connpy.grpc_layer.stubs import NodeStub
channel, remote_host = self._get_remote_channel()
node_stub = NodeStub(channel, remote_host=remote_host, config=self.config)
candidates = node_stub.list_nodes(filter_str=nodes_filter)
else:
candidates = (
self.config._getallnodes(nodes_filter)
if hasattr(self.config, &#34;_getallnodes&#34;)
else []
)
if nodes_filter:
cfg_dict = getattr(self.config, &#34;config&#34;, None)
case = cfg_dict.get(&#34;case&#34;, False) if isinstance(cfg_dict, dict) else False
flags = 0 if case else re.IGNORECASE
candidates = [c for c in candidates if re.search(nodes_filter, c, flags)]
permitted = ctx_svc.filter_node_list(result=candidates)
if not permitted:
raise ConnpyError(f&#34;No nodes found matching filter: {nodes_filter}&#34;)
return [f&#34;^{re.escape(n)}$&#34; for n in permitted]
def _build_remote_filter_batches(self, nodes_filter):
filtered = self._filter_remote_nodes_filter(nodes_filter)
if isinstance(filtered, list):
return [
filtered[i : i + MAX_REMOTE_FILTER_NODES]
for i in range(0, len(filtered), MAX_REMOTE_FILTER_NODES)
]
return [filtered]
@staticmethod
def _aggregate_remote_results(batch_results):
if len(batch_results) == 1:
return batch_results[0]
if all(isinstance(r, dict) for r in batch_results):
merged: dict[str, Any] = {}
for r in batch_results:
merged.update(r)
return merged
if all(isinstance(r, list) for r in batch_results):
merged_list: list[Any] = []
for r in batch_results:
merged_list.extend(r)
return merged_list
return batch_results[-1]
def run_commands(
self,
nodes_filter: str,
commands: List[str],
variables: Optional[Dict[str, Any]] = None,
commands: list[str],
variables: dict[str, Any] | None = None,
parallel: int = 10,
timeout: int = 20,
folder: Optional[str] = None,
prompt: Optional[str] = None,
on_node_complete: Optional[Callable] = None,
logger: Optional[Callable] = None,
name: Optional[str] = None
) -&gt; Dict[str, str]:
timeout: int | None = None,
folder: str | None = None,
prompt: str | None = None,
on_node_complete: Callable | None = None,
logger: Callable | None = None,
name: str | None = None,
connect_timeout: int | None = None,
auth_timeout: int | None = None,
) -&gt; dict[str, Any]:
&#34;&#34;&#34;Execute commands on a set of nodes.&#34;&#34;&#34;
try:
self._ensure_backend_supports_stage_timeouts(
connect_timeout=connect_timeout,
auth_timeout=auth_timeout,
)
if self._is_remote_backend():
batches = self._build_remote_filter_batches(nodes_filter)
stub = self._get_remote_stub()
batch_results = [
stub.run_commands(
nodes_filter=batch,
commands=commands,
variables=variables,
parallel=parallel,
timeout=timeout if timeout is not None else 20,
folder=folder,
prompt=prompt,
on_node_complete=on_node_complete,
logger=logger,
name=name or &#34;&#34;,
)
for batch in batches
]
return self._aggregate_remote_results(batch_results)
matched_names = self.config._getallnodes(nodes_filter)
ctx_svc = self._get_context_service()
if ctx_svc is not None and not ctx_svc.is_default_all_context():
matched_names = ctx_svc.filter_node_list(result=matched_names)
if not matched_names:
raise ConnpyError(f&#34;No nodes found matching filter: {nodes_filter}&#34;)
@@ -82,47 +255,96 @@ el.replaceWith(d);
executor = Nodes(node_data, config=self.config)
self.last_executor = executor
results = executor.run(
commands=commands,
vars=variables,
parallel=parallel,
timeout=timeout,
folder=folder,
prompt=prompt,
on_complete=on_node_complete,
logger=logger
)
run_kwargs: dict[str, Any] = {
&#34;commands&#34;: commands,
&#34;vars&#34;: variables,
&#34;parallel&#34;: parallel,
&#34;timeout&#34;: timeout,
&#34;folder&#34;: folder,
&#34;prompt&#34;: prompt,
&#34;on_complete&#34;: on_node_complete,
&#34;logger&#34;: logger,
&#34;connect_timeout&#34;: connect_timeout,
&#34;auth_timeout&#34;: auth_timeout,
&#34;_default_cmd_timeout&#34;: 20,
}
run_params = inspect.signature(executor.run).parameters
if &#34;unattended&#34; in run_params:
run_kwargs[&#34;unattended&#34;] = True
if &#34;attempt_tracker&#34; in run_params:
run_kwargs[&#34;attempt_tracker&#34;] = self.attempt_tracker
# Combine output and status for the caller
results = executor.run(**run_kwargs)
fb_map = getattr(executor, &#34;fallback_used&#34;, {})
fb_dict = fb_map if isinstance(fb_map, dict) else {}
# Combine output, status, and fallback metadata for the caller
full_results = {}
self.last_fallback_used = {}
for unique in results:
fb_used = bool(fb_dict.get(unique, False))
full_results[unique] = {
&#34;output&#34;: results[unique],
&#34;status&#34;: executor.status.get(unique, 1)
&#34;status&#34;: executor.status.get(unique, 1),
&#34;fallback_used&#34;: fb_used,
}
self.last_fallback_used[unique] = fb_used
return full_results
except Exception as e:
raise ConnpyError(f&#34;Execution failed: {e}&#34;)
except ConnpyError:
raise
except Exception as e: # noqa: BLE001
raise_sanitized_auth_error(e, code=&#34;AUTH_FAILED&#34;, context_label=&#34;execution failed&#34;)
def test_commands(
self,
nodes_filter: str,
commands: List[str],
expected: List[str],
variables: Optional[Dict[str, Any]] = None,
commands: list[str],
expected: list[str],
variables: dict[str, Any] | None = None,
parallel: int = 10,
timeout: int = 20,
folder: Optional[str] = None,
prompt: Optional[str] = None,
on_node_complete: Optional[Callable] = None,
logger: Optional[Callable] = None,
name: Optional[str] = None
) -&gt; Dict[str, Dict[str, bool]]:
timeout: int | None = None,
folder: str | None = None,
prompt: str | None = None,
on_node_complete: Callable | None = None,
logger: Callable | None = None,
name: str | None = None,
connect_timeout: int | None = None,
auth_timeout: int | None = None,
) -&gt; dict[str, dict[str, bool]]:
&#34;&#34;&#34;Run commands and verify expected output on a set of nodes.&#34;&#34;&#34;
try:
self._ensure_backend_supports_stage_timeouts(
connect_timeout=connect_timeout,
auth_timeout=auth_timeout,
)
if self._is_remote_backend():
batches = self._build_remote_filter_batches(nodes_filter)
stub = self._get_remote_stub()
batch_results = [
stub.test_commands(
nodes_filter=batch,
commands=commands,
expected=expected,
variables=variables,
parallel=parallel,
timeout=timeout if timeout is not None else 20,
folder=folder or &#34;&#34;,
prompt=prompt,
on_node_complete=on_node_complete,
logger=logger,
name=name or &#34;&#34;,
)
for batch in batches
]
return self._aggregate_remote_results(batch_results)
matched_names = self.config._getallnodes(nodes_filter)
ctx_svc = self._get_context_service()
if ctx_svc is not None and not ctx_svc.is_default_all_context():
matched_names = ctx_svc.filter_node_list(result=matched_names)
if not matched_names:
raise ConnpyError(f&#34;No nodes found matching filter: {nodes_filter}&#34;)
@@ -130,38 +352,57 @@ el.replaceWith(d);
executor = Nodes(node_data, config=self.config)
self.last_executor = executor
results = executor.test(
commands=commands,
expected=expected,
vars=variables,
parallel=parallel,
timeout=timeout,
folder=folder,
prompt=prompt,
on_complete=on_node_complete,
logger=logger
)
return results
except Exception as e:
raise ConnpyError(f&#34;Testing failed: {e}&#34;)
test_kwargs: dict[str, Any] = {
&#34;commands&#34;: commands,
&#34;expected&#34;: expected,
&#34;vars&#34;: variables,
&#34;parallel&#34;: parallel,
&#34;timeout&#34;: timeout,
&#34;folder&#34;: folder,
&#34;prompt&#34;: prompt,
&#34;on_complete&#34;: on_node_complete,
&#34;logger&#34;: logger,
&#34;connect_timeout&#34;: connect_timeout,
&#34;auth_timeout&#34;: auth_timeout,
&#34;_default_cmd_timeout&#34;: 20,
}
test_params = inspect.signature(executor.test).parameters
if &#34;unattended&#34; in test_params:
test_kwargs[&#34;unattended&#34;] = True
if &#34;attempt_tracker&#34; in test_params:
test_kwargs[&#34;attempt_tracker&#34;] = self.attempt_tracker
def run_cli_script(self, nodes_filter: str, script_path: str, parallel: int = 10) -&gt; Dict[str, str]:
res = executor.test(**test_kwargs)
fb_map = getattr(executor, &#34;fallback_used&#34;, {})
fb_dict = fb_map if isinstance(fb_map, dict) else {}
if isinstance(res, dict):
self.last_fallback_used = {u: bool(fb_dict.get(u, False)) for u in res}
else:
self.last_fallback_used = {}
return res
except ConnpyError:
raise
except Exception as e: # noqa: BLE001
raise_sanitized_auth_error(e, code=&#34;AUTH_FAILED&#34;, context_label=&#34;testing failed&#34;)
def run_cli_script(self, nodes_filter: str, script_path: str, parallel: int = 10) -&gt; dict[str, str]:
&#34;&#34;&#34;Run a plain-text script containing one command per line.&#34;&#34;&#34;
if not os.path.exists(script_path):
raise ConnpyError(f&#34;Script file not found: {script_path}&#34;)
try:
with open(script_path, &#34;r&#34;) as f:
with open(script_path) as f:
commands = [line.strip() for line in f if line.strip()]
except Exception as e:
raise ConnpyError(f&#34;Failed to read script {script_path}: {e}&#34;)
except OSError:
raise ConnpyError(&#34;Failed to read script file&#34;) from None
return self.run_commands(nodes_filter, commands, parallel=parallel)</code></pre>
</details>
<div class="desc"><p>Business logic for executing commands on nodes and running automation scripts.</p>
<p>Initialize the service.</p>
<h2 id="args">Args</h2>
<dl>
<dt>Args</dt>
<dt>-----=</dt>
<dt><strong><code>config</code></strong></dt>
<dd>An instance of configfile (or None to instantiate a new one/use global context).</dd>
</dl></div>
@@ -172,30 +413,30 @@ el.replaceWith(d);
<h3>Methods</h3>
<dl>
<dt id="connpy.services.execution_service.ExecutionService.run_cli_script"><code class="name flex">
<span>def <span class="ident">run_cli_script</span></span>(<span>self, nodes_filter: str, script_path: str, parallel: int = 10) ‑> Dict[str, str]</span>
<span>def <span class="ident">run_cli_script</span></span>(<span>self, nodes_filter: str, script_path: str, parallel: int = 10) ‑> dict[str, str]</span>
</code></dt>
<dd>
<details class="source">
<summary>
<span>Expand source code</span>
</summary>
<pre><code class="python">def run_cli_script(self, nodes_filter: str, script_path: str, parallel: int = 10) -&gt; Dict[str, str]:
<pre><code class="python">def run_cli_script(self, nodes_filter: str, script_path: str, parallel: int = 10) -&gt; dict[str, str]:
&#34;&#34;&#34;Run a plain-text script containing one command per line.&#34;&#34;&#34;
if not os.path.exists(script_path):
raise ConnpyError(f&#34;Script file not found: {script_path}&#34;)
try:
with open(script_path, &#34;r&#34;) as f:
with open(script_path) as f:
commands = [line.strip() for line in f if line.strip()]
except Exception as e:
raise ConnpyError(f&#34;Failed to read script {script_path}: {e}&#34;)
except OSError:
raise ConnpyError(&#34;Failed to read script file&#34;) from None
return self.run_commands(nodes_filter, commands, parallel=parallel)</code></pre>
</details>
<div class="desc"><p>Run a plain-text script containing one command per line.</p></div>
</dd>
<dt id="connpy.services.execution_service.ExecutionService.run_commands"><code class="name flex">
<span>def <span class="ident">run_commands</span></span>(<span>self,<br>nodes_filter: str,<br>commands: List[str],<br>variables: Dict[str, Any] | None = None,<br>parallel: int = 10,<br>timeout: int = 20,<br>folder: str | None = None,<br>prompt: str | None = None,<br>on_node_complete: Callable | None = None,<br>logger: Callable | None = None,<br>name: str | None = None) ‑> Dict[str, str]</span>
<span>def <span class="ident">run_commands</span></span>(<span>self,<br>nodes_filter: str,<br>commands: list[str],<br>variables: dict[str, typing.Any] | None = None,<br>parallel: int = 10,<br>timeout: int | None = None,<br>folder: str | None = None,<br>prompt: str | None = None,<br>on_node_complete: collections.abc.Callable | None = None,<br>logger: collections.abc.Callable | None = None,<br>name: str | None = None,<br>connect_timeout: int | None = None,<br>auth_timeout: int | None = None) ‑> dict[str, typing.Any]</span>
</code></dt>
<dd>
<details class="source">
@@ -205,20 +446,49 @@ el.replaceWith(d);
<pre><code class="python">def run_commands(
self,
nodes_filter: str,
commands: List[str],
variables: Optional[Dict[str, Any]] = None,
commands: list[str],
variables: dict[str, Any] | None = None,
parallel: int = 10,
timeout: int = 20,
folder: Optional[str] = None,
prompt: Optional[str] = None,
on_node_complete: Optional[Callable] = None,
logger: Optional[Callable] = None,
name: Optional[str] = None
) -&gt; Dict[str, str]:
timeout: int | None = None,
folder: str | None = None,
prompt: str | None = None,
on_node_complete: Callable | None = None,
logger: Callable | None = None,
name: str | None = None,
connect_timeout: int | None = None,
auth_timeout: int | None = None,
) -&gt; dict[str, Any]:
&#34;&#34;&#34;Execute commands on a set of nodes.&#34;&#34;&#34;
try:
self._ensure_backend_supports_stage_timeouts(
connect_timeout=connect_timeout,
auth_timeout=auth_timeout,
)
if self._is_remote_backend():
batches = self._build_remote_filter_batches(nodes_filter)
stub = self._get_remote_stub()
batch_results = [
stub.run_commands(
nodes_filter=batch,
commands=commands,
variables=variables,
parallel=parallel,
timeout=timeout if timeout is not None else 20,
folder=folder,
prompt=prompt,
on_node_complete=on_node_complete,
logger=logger,
name=name or &#34;&#34;,
)
for batch in batches
]
return self._aggregate_remote_results(batch_results)
matched_names = self.config._getallnodes(nodes_filter)
ctx_svc = self._get_context_service()
if ctx_svc is not None and not ctx_svc.is_default_all_context():
matched_names = ctx_svc.filter_node_list(result=matched_names)
if not matched_names:
raise ConnpyError(f&#34;No nodes found matching filter: {nodes_filter}&#34;)
@@ -226,33 +496,52 @@ el.replaceWith(d);
executor = Nodes(node_data, config=self.config)
self.last_executor = executor
results = executor.run(
commands=commands,
vars=variables,
parallel=parallel,
timeout=timeout,
folder=folder,
prompt=prompt,
on_complete=on_node_complete,
logger=logger
)
run_kwargs: dict[str, Any] = {
&#34;commands&#34;: commands,
&#34;vars&#34;: variables,
&#34;parallel&#34;: parallel,
&#34;timeout&#34;: timeout,
&#34;folder&#34;: folder,
&#34;prompt&#34;: prompt,
&#34;on_complete&#34;: on_node_complete,
&#34;logger&#34;: logger,
&#34;connect_timeout&#34;: connect_timeout,
&#34;auth_timeout&#34;: auth_timeout,
&#34;_default_cmd_timeout&#34;: 20,
}
run_params = inspect.signature(executor.run).parameters
if &#34;unattended&#34; in run_params:
run_kwargs[&#34;unattended&#34;] = True
if &#34;attempt_tracker&#34; in run_params:
run_kwargs[&#34;attempt_tracker&#34;] = self.attempt_tracker
# Combine output and status for the caller
results = executor.run(**run_kwargs)
fb_map = getattr(executor, &#34;fallback_used&#34;, {})
fb_dict = fb_map if isinstance(fb_map, dict) else {}
# Combine output, status, and fallback metadata for the caller
full_results = {}
self.last_fallback_used = {}
for unique in results:
fb_used = bool(fb_dict.get(unique, False))
full_results[unique] = {
&#34;output&#34;: results[unique],
&#34;status&#34;: executor.status.get(unique, 1)
&#34;status&#34;: executor.status.get(unique, 1),
&#34;fallback_used&#34;: fb_used,
}
self.last_fallback_used[unique] = fb_used
return full_results
except Exception as e:
raise ConnpyError(f&#34;Execution failed: {e}&#34;)</code></pre>
except ConnpyError:
raise
except Exception as e: # noqa: BLE001
raise_sanitized_auth_error(e, code=&#34;AUTH_FAILED&#34;, context_label=&#34;execution failed&#34;)</code></pre>
</details>
<div class="desc"><p>Execute commands on a set of nodes.</p></div>
</dd>
<dt id="connpy.services.execution_service.ExecutionService.test_commands"><code class="name flex">
<span>def <span class="ident">test_commands</span></span>(<span>self,<br>nodes_filter: str,<br>commands: List[str],<br>expected: List[str],<br>variables: Dict[str, Any] | None = None,<br>parallel: int = 10,<br>timeout: int = 20,<br>folder: str | None = None,<br>prompt: str | None = None,<br>on_node_complete: Callable | None = None,<br>logger: Callable | None = None,<br>name: str | None = None) ‑> Dict[str, Dict[str, bool]]</span>
<span>def <span class="ident">test_commands</span></span>(<span>self,<br>nodes_filter: str,<br>commands: list[str],<br>expected: list[str],<br>variables: dict[str, typing.Any] | None = None,<br>parallel: int = 10,<br>timeout: int | None = None,<br>folder: str | None = None,<br>prompt: str | None = None,<br>on_node_complete: collections.abc.Callable | None = None,<br>logger: collections.abc.Callable | None = None,<br>name: str | None = None,<br>connect_timeout: int | None = None,<br>auth_timeout: int | None = None) ‑> dict[str, dict[str, bool]]</span>
</code></dt>
<dd>
<details class="source">
@@ -262,21 +551,51 @@ el.replaceWith(d);
<pre><code class="python">def test_commands(
self,
nodes_filter: str,
commands: List[str],
expected: List[str],
variables: Optional[Dict[str, Any]] = None,
commands: list[str],
expected: list[str],
variables: dict[str, Any] | None = None,
parallel: int = 10,
timeout: int = 20,
folder: Optional[str] = None,
prompt: Optional[str] = None,
on_node_complete: Optional[Callable] = None,
logger: Optional[Callable] = None,
name: Optional[str] = None
) -&gt; Dict[str, Dict[str, bool]]:
timeout: int | None = None,
folder: str | None = None,
prompt: str | None = None,
on_node_complete: Callable | None = None,
logger: Callable | None = None,
name: str | None = None,
connect_timeout: int | None = None,
auth_timeout: int | None = None,
) -&gt; dict[str, dict[str, bool]]:
&#34;&#34;&#34;Run commands and verify expected output on a set of nodes.&#34;&#34;&#34;
try:
self._ensure_backend_supports_stage_timeouts(
connect_timeout=connect_timeout,
auth_timeout=auth_timeout,
)
if self._is_remote_backend():
batches = self._build_remote_filter_batches(nodes_filter)
stub = self._get_remote_stub()
batch_results = [
stub.test_commands(
nodes_filter=batch,
commands=commands,
expected=expected,
variables=variables,
parallel=parallel,
timeout=timeout if timeout is not None else 20,
folder=folder or &#34;&#34;,
prompt=prompt,
on_node_complete=on_node_complete,
logger=logger,
name=name or &#34;&#34;,
)
for batch in batches
]
return self._aggregate_remote_results(batch_results)
matched_names = self.config._getallnodes(nodes_filter)
ctx_svc = self._get_context_service()
if ctx_svc is not None and not ctx_svc.is_default_all_context():
matched_names = ctx_svc.filter_node_list(result=matched_names)
if not matched_names:
raise ConnpyError(f&#34;No nodes found matching filter: {nodes_filter}&#34;)
@@ -284,20 +603,38 @@ el.replaceWith(d);
executor = Nodes(node_data, config=self.config)
self.last_executor = executor
results = executor.test(
commands=commands,
expected=expected,
vars=variables,
parallel=parallel,
timeout=timeout,
folder=folder,
prompt=prompt,
on_complete=on_node_complete,
logger=logger
)
return results
except Exception as e:
raise ConnpyError(f&#34;Testing failed: {e}&#34;)</code></pre>
test_kwargs: dict[str, Any] = {
&#34;commands&#34;: commands,
&#34;expected&#34;: expected,
&#34;vars&#34;: variables,
&#34;parallel&#34;: parallel,
&#34;timeout&#34;: timeout,
&#34;folder&#34;: folder,
&#34;prompt&#34;: prompt,
&#34;on_complete&#34;: on_node_complete,
&#34;logger&#34;: logger,
&#34;connect_timeout&#34;: connect_timeout,
&#34;auth_timeout&#34;: auth_timeout,
&#34;_default_cmd_timeout&#34;: 20,
}
test_params = inspect.signature(executor.test).parameters
if &#34;unattended&#34; in test_params:
test_kwargs[&#34;unattended&#34;] = True
if &#34;attempt_tracker&#34; in test_params:
test_kwargs[&#34;attempt_tracker&#34;] = self.attempt_tracker
res = executor.test(**test_kwargs)
fb_map = getattr(executor, &#34;fallback_used&#34;, {})
fb_dict = fb_map if isinstance(fb_map, dict) else {}
if isinstance(res, dict):
self.last_fallback_used = {u: bool(fb_dict.get(u, False)) for u in res}
else:
self.last_fallback_used = {}
return res
except ConnpyError:
raise
except Exception as e: # noqa: BLE001
raise_sanitized_auth_error(e, code=&#34;AUTH_FAILED&#34;, context_label=&#34;testing failed&#34;)</code></pre>
</details>
<div class="desc"><p>Run commands and verify expected output on a set of nodes.</p></div>
</dd>
+248 -53
View File
@@ -48,7 +48,7 @@ el.replaceWith(d);
<dl>
<dt id="connpy.services.import_export_service.ImportExportService"><code class="flex name class">
<span>class <span class="ident">ImportExportService</span></span>
<span>(</span><span>config=None)</span>
<span>(</span><span>config=None, enforce_context: bool = True)</span>
</code></dt>
<dd>
<details class="source">
@@ -58,6 +58,44 @@ el.replaceWith(d);
<pre><code class="python">class ImportExportService(BaseService):
&#34;&#34;&#34;Business logic for YAML/JSON inventory import and export.&#34;&#34;&#34;
def __init__(self, config=None, enforce_context: bool = True):
super().__init__(config)
self.enforce_context = bool(enforce_context)
def _get_context_service(self, cfg=None):
target_cfg = cfg or self.config
if (
not getattr(self, &#34;enforce_context&#34;, True)
or not getattr(target_cfg, &#34;_enforce_context&#34;, True)
or (len(sys.argv) &gt; 1 and sys.argv[1] == &#34;api&#34;)
or not isinstance(getattr(target_cfg, &#34;config&#34;, None), dict)
):
return None
from .context_service import ContextService
return ContextService(target_cfg)
def _get_raw_folders(self, cfg=None) -&gt; list[str]:
target_cfg = cfg or self.config
if target_cfg is None:
return []
if (
not hasattr(target_cfg, &#34;assert_called&#34;)
and callable(getattr(target_cfg, &#34;_getallfolders_unlocked&#34;, None))
and hasattr(target_cfg, &#34;_mutation_lock&#34;)
):
with target_cfg._mutation_lock:
return target_cfg._getallfolders_unlocked()
return target_cfg._getallfolders()
def _folder_exists(self, folder_id: str, raw_folders: list[str], cfg=None) -&gt; bool:
target_cfg = cfg or self.config
cfg_dict = getattr(target_cfg, &#34;config&#34;, None)
case = cfg_dict.get(&#34;case&#34;, False) if isinstance(cfg_dict, dict) else False
if case:
return folder_id in raw_folders
return folder_id.lower() in {f.lower() for f in raw_folders}
def export_to_file(self, file_path, folders=None):
&#34;&#34;&#34;Export nodes/folders to a YAML file.&#34;&#34;&#34;
if os.path.exists(file_path):
@@ -72,16 +110,42 @@ el.replaceWith(d);
def export_to_dict(self, folders=None):
&#34;&#34;&#34;Export nodes/folders to a dictionary.&#34;&#34;&#34;
if not folders:
return deepcopy(self.config.connections)
else:
# Validate folders exist
for f in folders:
if f != &#34;@&#34; and f not in self.config._getallfolders():
raise NodeNotFoundError(f&#34;Folder &#39;{f}&#39; not found.&#34;)
ctx_svc = self._get_context_service()
ctx_active = ctx_svc is not None and not ctx_svc.is_default_all_context()
if not folders and not ctx_active:
return deepcopy(self.config.connections)
raw_folders = self._get_raw_folders()
if folders:
for f in folders:
if f != &#34;@&#34; and (
not self._folder_exists(f, raw_folders)
or (ctx_active and not ctx_svc.matches_active_context(f))
):
raise NodeNotFoundError(f&#34;Folder &#39;{f}&#39; not found.&#34;)
flat = self.config._getallnodesfull(folders, extract=False)
allowed_folders = []
else:
allowed_folders = ctx_svc.filter_node_list(result=raw_folders)
flat = self.config._getallnodesfull(extract=False)
if ctx_active:
flat = ctx_svc.filter_node_dict(result=flat)
nested = {}
for folder_id in allowed_folders:
uniques = self.config._explode_unique(folder_id)
if not uniques or &#34;folder&#34; not in uniques:
continue
f_name = uniques[&#34;folder&#34;]
if f_name not in nested:
nested[f_name] = {&#34;type&#34;: &#34;folder&#34;}
if &#34;subfolder&#34; in uniques:
s_name = uniques[&#34;subfolder&#34;]
if s_name not in nested[f_name]:
nested[f_name][s_name] = {&#34;type&#34;: &#34;subfolder&#34;}
for k, v in flat.items():
uniques = self.config._explode_unique(k)
if not uniques:
@@ -120,28 +184,58 @@ el.replaceWith(d);
try:
with open(file_path, &#34;r&#34;) as f:
data = yaml.load(f, Loader=yaml.FullLoader)
data = yaml.safe_load(f)
self.import_from_dict(data)
except Exception as e:
raise InvalidConfigurationError(f&#34;Failed to read/parse import file: {e}&#34;)
def import_from_dict(self, data):
&#34;&#34;&#34;Import nodes/folders from a dictionary.&#34;&#34;&#34;
&#34;&#34;&#34;Import nodes/folders from a dictionary atomically with context pre-validation.&#34;&#34;&#34;
if not isinstance(data, dict):
raise InvalidConfigurationError(&#34;Invalid import data format: expected a dictionary of nodes.&#34;)
raise InvalidConfigurationError(
&#34;Invalid import data format: expected a dictionary of nodes.&#34;
)
def _traverse_import(node_data, current_folder=&#39;&#39;, current_subfolder=&#39;&#39;):
ctx_svc = self._get_context_service()
ctx_active = ctx_svc is not None and not ctx_svc.is_default_all_context()
raw_folders = self._get_raw_folders() if ctx_active else []
planned_ops = []
def _plan_import(node_data, current_folder=&#34;&#34;, current_subfolder=&#34;&#34;):
for k, v in node_data.items():
if k == &#34;type&#34;:
if k == &#34;type&#34; or not isinstance(v, dict):
continue
if isinstance(v, dict):
node_type = v.get(&#34;type&#34;, &#34;connection&#34;)
if node_type == &#34;folder&#34;:
self.config._folder_add(folder=k)
_traverse_import(v, current_folder=k, current_subfolder=&#39;&#39;)
folder_id = f&#34;@{k}&#34;
has_children = any(
isinstance(cv, dict) for ck, cv in v.items() if ck != &#34;type&#34;
)
if ctx_active and (
not has_children
or (
not ctx_svc.matches_active_context(folder_id)
and not self._folder_exists(folder_id, raw_folders)
)
):
ctx_svc.validate_in_active_context(folder_id, is_folder=True)
planned_ops.append((&#34;folder&#34;, k, &#34;&#34;))
_plan_import(v, current_folder=k, current_subfolder=&#34;&#34;)
elif node_type == &#34;subfolder&#34;:
self.config._folder_add(folder=current_folder, subfolder=k)
_traverse_import(v, current_folder=current_folder, current_subfolder=k)
folder_id = f&#34;@{k}@{current_folder}&#34;
has_children = any(
isinstance(cv, dict) for ck, cv in v.items() if ck != &#34;type&#34;
)
if ctx_active and (
not has_children
or (
not ctx_svc.matches_active_context(folder_id)
and not self._folder_exists(folder_id, raw_folders)
)
):
ctx_svc.validate_in_active_context(folder_id, is_folder=True)
planned_ops.append((&#34;folder&#34;, current_folder, k))
_plan_import(v, current_folder=current_folder, current_subfolder=k)
elif node_type == &#34;connection&#34;:
unique_id = k
if current_subfolder:
@@ -149,24 +243,47 @@ el.replaceWith(d);
elif current_folder:
unique_id = f&#34;{k}@{current_folder}&#34;
self._validate_node_name(unique_id)
NodeService._validate_node_payload(v)
if ctx_svc is not None:
ctx_svc.validate_in_active_context(unique_id, is_folder=False)
kwargs = deepcopy(v)
kwargs[&#39;id&#39;] = k
kwargs[&#39;folder&#39;] = current_folder
kwargs[&#39;subfolder&#39;] = current_subfolder
kwargs[&#34;id&#34;] = k
kwargs[&#34;folder&#34;] = current_folder
kwargs[&#34;subfolder&#34;] = current_subfolder
planned_ops.append((&#34;connection&#34;, kwargs))
self.config._connections_add(**kwargs)
_plan_import(data)
if _supports_transactional_update(self.config):
def _mut(cfg):
for op in planned_ops:
if op[0] == &#34;folder&#34;:
cfg._folder_add_unlocked(folder=op[1], subfolder=op[2])
else:
# Invalid format skip
pass
cfg._connections_add_unlocked(**op[1])
_traverse_import(data)
self.config._saveconfig(self.config.file)</code></pre>
self.config.transactional_update(_mut)
return
snap_connections = deepcopy(self.config.connections)
try:
for op in planned_ops:
if op[0] == &#34;folder&#34;:
self.config._folder_add(folder=op[1], subfolder=op[2])
else:
self.config._connections_add(**op[1])
self.config._saveconfig(self.config.file)
except BaseException:
self.config.connections.clear()
self.config.connections.update(snap_connections)
raise</code></pre>
</details>
<div class="desc"><p>Business logic for YAML/JSON inventory import and export.</p>
<p>Initialize the service.</p>
<h2 id="args">Args</h2>
<dl>
<dt>Args</dt>
<dt>-----=</dt>
<dt><strong><code>config</code></strong></dt>
<dd>An instance of configfile (or None to instantiate a new one/use global context).</dd>
</dl></div>
@@ -186,16 +303,42 @@ el.replaceWith(d);
</summary>
<pre><code class="python">def export_to_dict(self, folders=None):
&#34;&#34;&#34;Export nodes/folders to a dictionary.&#34;&#34;&#34;
if not folders:
return deepcopy(self.config.connections)
else:
# Validate folders exist
for f in folders:
if f != &#34;@&#34; and f not in self.config._getallfolders():
raise NodeNotFoundError(f&#34;Folder &#39;{f}&#39; not found.&#34;)
ctx_svc = self._get_context_service()
ctx_active = ctx_svc is not None and not ctx_svc.is_default_all_context()
if not folders and not ctx_active:
return deepcopy(self.config.connections)
raw_folders = self._get_raw_folders()
if folders:
for f in folders:
if f != &#34;@&#34; and (
not self._folder_exists(f, raw_folders)
or (ctx_active and not ctx_svc.matches_active_context(f))
):
raise NodeNotFoundError(f&#34;Folder &#39;{f}&#39; not found.&#34;)
flat = self.config._getallnodesfull(folders, extract=False)
allowed_folders = []
else:
allowed_folders = ctx_svc.filter_node_list(result=raw_folders)
flat = self.config._getallnodesfull(extract=False)
if ctx_active:
flat = ctx_svc.filter_node_dict(result=flat)
nested = {}
for folder_id in allowed_folders:
uniques = self.config._explode_unique(folder_id)
if not uniques or &#34;folder&#34; not in uniques:
continue
f_name = uniques[&#34;folder&#34;]
if f_name not in nested:
nested[f_name] = {&#34;type&#34;: &#34;folder&#34;}
if &#34;subfolder&#34; in uniques:
s_name = uniques[&#34;subfolder&#34;]
if s_name not in nested[f_name]:
nested[f_name][s_name] = {&#34;type&#34;: &#34;subfolder&#34;}
for k, v in flat.items():
uniques = self.config._explode_unique(k)
if not uniques:
@@ -260,22 +403,52 @@ el.replaceWith(d);
<span>Expand source code</span>
</summary>
<pre><code class="python">def import_from_dict(self, data):
&#34;&#34;&#34;Import nodes/folders from a dictionary.&#34;&#34;&#34;
&#34;&#34;&#34;Import nodes/folders from a dictionary atomically with context pre-validation.&#34;&#34;&#34;
if not isinstance(data, dict):
raise InvalidConfigurationError(&#34;Invalid import data format: expected a dictionary of nodes.&#34;)
raise InvalidConfigurationError(
&#34;Invalid import data format: expected a dictionary of nodes.&#34;
)
def _traverse_import(node_data, current_folder=&#39;&#39;, current_subfolder=&#39;&#39;):
ctx_svc = self._get_context_service()
ctx_active = ctx_svc is not None and not ctx_svc.is_default_all_context()
raw_folders = self._get_raw_folders() if ctx_active else []
planned_ops = []
def _plan_import(node_data, current_folder=&#34;&#34;, current_subfolder=&#34;&#34;):
for k, v in node_data.items():
if k == &#34;type&#34;:
if k == &#34;type&#34; or not isinstance(v, dict):
continue
if isinstance(v, dict):
node_type = v.get(&#34;type&#34;, &#34;connection&#34;)
if node_type == &#34;folder&#34;:
self.config._folder_add(folder=k)
_traverse_import(v, current_folder=k, current_subfolder=&#39;&#39;)
folder_id = f&#34;@{k}&#34;
has_children = any(
isinstance(cv, dict) for ck, cv in v.items() if ck != &#34;type&#34;
)
if ctx_active and (
not has_children
or (
not ctx_svc.matches_active_context(folder_id)
and not self._folder_exists(folder_id, raw_folders)
)
):
ctx_svc.validate_in_active_context(folder_id, is_folder=True)
planned_ops.append((&#34;folder&#34;, k, &#34;&#34;))
_plan_import(v, current_folder=k, current_subfolder=&#34;&#34;)
elif node_type == &#34;subfolder&#34;:
self.config._folder_add(folder=current_folder, subfolder=k)
_traverse_import(v, current_folder=current_folder, current_subfolder=k)
folder_id = f&#34;@{k}@{current_folder}&#34;
has_children = any(
isinstance(cv, dict) for ck, cv in v.items() if ck != &#34;type&#34;
)
if ctx_active and (
not has_children
or (
not ctx_svc.matches_active_context(folder_id)
and not self._folder_exists(folder_id, raw_folders)
)
):
ctx_svc.validate_in_active_context(folder_id, is_folder=True)
planned_ops.append((&#34;folder&#34;, current_folder, k))
_plan_import(v, current_folder=current_folder, current_subfolder=k)
elif node_type == &#34;connection&#34;:
unique_id = k
if current_subfolder:
@@ -283,21 +456,43 @@ el.replaceWith(d);
elif current_folder:
unique_id = f&#34;{k}@{current_folder}&#34;
self._validate_node_name(unique_id)
NodeService._validate_node_payload(v)
if ctx_svc is not None:
ctx_svc.validate_in_active_context(unique_id, is_folder=False)
kwargs = deepcopy(v)
kwargs[&#39;id&#39;] = k
kwargs[&#39;folder&#39;] = current_folder
kwargs[&#39;subfolder&#39;] = current_subfolder
kwargs[&#34;id&#34;] = k
kwargs[&#34;folder&#34;] = current_folder
kwargs[&#34;subfolder&#34;] = current_subfolder
planned_ops.append((&#34;connection&#34;, kwargs))
self.config._connections_add(**kwargs)
_plan_import(data)
if _supports_transactional_update(self.config):
def _mut(cfg):
for op in planned_ops:
if op[0] == &#34;folder&#34;:
cfg._folder_add_unlocked(folder=op[1], subfolder=op[2])
else:
# Invalid format skip
pass
cfg._connections_add_unlocked(**op[1])
_traverse_import(data)
self.config._saveconfig(self.config.file)</code></pre>
self.config.transactional_update(_mut)
return
snap_connections = deepcopy(self.config.connections)
try:
for op in planned_ops:
if op[0] == &#34;folder&#34;:
self.config._folder_add(folder=op[1], subfolder=op[2])
else:
self.config._connections_add(**op[1])
self.config._saveconfig(self.config.file)
except BaseException:
self.config.connections.clear()
self.config.connections.update(snap_connections)
raise</code></pre>
</details>
<div class="desc"><p>Import nodes/folders from a dictionary.</p></div>
<div class="desc"><p>Import nodes/folders from a dictionary atomically with context pre-validation.</p></div>
</dd>
<dt id="connpy.services.import_export_service.ImportExportService.import_from_file"><code class="name flex">
<span>def <span class="ident">import_from_file</span></span>(<span>self, file_path)</span>
@@ -314,7 +509,7 @@ el.replaceWith(d);
try:
with open(file_path, &#34;r&#34;) as f:
data = yaml.load(f, Loader=yaml.FullLoader)
data = yaml.safe_load(f)
self.import_from_dict(data)
except Exception as e:
raise InvalidConfigurationError(f&#34;Failed to read/parse import file: {e}&#34;)</code></pre>
File diff suppressed because it is too large. Load diff
+581 -63
View File
@@ -48,7 +48,7 @@ el.replaceWith(d);
<dl>
<dt id="connpy.services.node_service.NodeService"><code class="flex name class">
<span>class <span class="ident">NodeService</span></span>
<span>(</span><span>config=None)</span>
<span>(</span><span>config=None, enforce_context: bool = True)</span>
</code></dt>
<dd>
<details class="source">
@@ -56,8 +56,148 @@ el.replaceWith(d);
<span>Expand source code</span>
</summary>
<pre><code class="python">class NodeService(BaseService):
def __init__(self, config=None):
_CONNECTION_FIELDS: ClassVar[frozenset[str]] = frozenset({
&#34;id&#34;,
&#34;host&#34;,
&#34;folder&#34;,
&#34;subfolder&#34;,
&#34;options&#34;,
&#34;logs&#34;,
&#34;password&#34;,
&#34;port&#34;,
&#34;protocol&#34;,
&#34;user&#34;,
&#34;tags&#34;,
&#34;jumphost&#34;,
&#34;fallback&#34;,
&#34;type&#34;,
})
def __init__(self, config=None, enforce_context: bool = True):
super().__init__(config)
self.enforce_context = bool(enforce_context)
def _get_context_service(self, cfg=None):
target_cfg = cfg or self.config
if (
not getattr(self, &#34;enforce_context&#34;, True)
or not getattr(target_cfg, &#34;_enforce_context&#34;, True)
or (len(sys.argv) &gt; 1 and sys.argv[1] == &#34;api&#34;)
or not isinstance(getattr(target_cfg, &#34;config&#34;, None), dict)
):
return None
from .context_service import ContextService
return ContextService(target_cfg)
def _get_raw_folders(self, cfg=None) -&gt; list[str]:
target_cfg = cfg or self.config
if target_cfg is None:
return []
if (
not hasattr(target_cfg, &#34;assert_called&#34;)
and callable(getattr(target_cfg, &#34;_getallfolders_unlocked&#34;, None))
and hasattr(target_cfg, &#34;_mutation_lock&#34;)
):
with target_cfg._mutation_lock:
return target_cfg._getallfolders_unlocked()
return target_cfg._getallfolders()
def _get_raw_nodes(self, cfg=None) -&gt; list[str]:
target_cfg = cfg or self.config
if target_cfg is None:
return []
if (
not hasattr(target_cfg, &#34;assert_called&#34;)
and callable(getattr(target_cfg, &#34;_getallnodes_unlocked&#34;, None))
and hasattr(target_cfg, &#34;_mutation_lock&#34;)
):
with target_cfg._mutation_lock:
return target_cfg._getallnodes_unlocked()
return target_cfg._getallnodes()
def _derive_parent_folder(self, cfg, unique_id: str, is_folder: bool) -&gt; str | None:
explode_fn = (
cfg._explode_unique_unlocked
if (
not hasattr(cfg, &#34;assert_called&#34;)
and callable(getattr(cfg, &#34;_explode_unique_unlocked&#34;, None))
)
else cfg._explode_unique
)
uniques = explode_fn(unique_id)
if is_folder:
if (
not isinstance(unique_id, str)
or not unique_id.startswith(&#34;@&#34;)
or not isinstance(uniques, dict)
or &#34;folder&#34; not in uniques
):
raise InvalidConfigurationError(f&#34;Invalid folder name &#39;{unique_id}&#39;.&#34;)
if &#34;subfolder&#34; in uniques:
return f&#34;@{uniques[&#39;folder&#39;]}&#34;
return None
if (
not isinstance(unique_id, str)
or unique_id.startswith(&#34;@&#34;)
or not isinstance(uniques, dict)
or not uniques.get(&#34;id&#34;)
):
raise InvalidConfigurationError(f&#34;Invalid node name &#39;{unique_id}&#39;.&#34;)
if &#34;subfolder&#34; in uniques and &#34;folder&#34; in uniques:
return f&#34;@{uniques[&#39;subfolder&#39;]}@{uniques[&#39;folder&#39;]}&#34;
if &#34;folder&#34; in uniques:
return f&#34;@{uniques[&#39;folder&#39;]}&#34;
return None
def _folder_exists(self, parent_folder: str, raw_folders: list[str], cfg=None) -&gt; bool:
target_cfg = cfg or self.config
cfg_dict = getattr(target_cfg, &#34;config&#34;, None)
case = cfg_dict.get(&#34;case&#34;, False) if isinstance(cfg_dict, dict) else False
if case:
return parent_folder in raw_folders
return parent_folder.lower() in {f.lower() for f in raw_folders}
def _ensure_in_active_context(
self, unique_id: str, *, is_folder: bool = False, not_found_msg: str | None = None
) -&gt; None:
ctx_svc = self._get_context_service()
if (
ctx_svc is not None
and not ctx_svc.is_default_all_context()
and not ctx_svc.matches_active_context(unique_id)
):
if not_found_msg is not None:
raise NodeNotFoundError(not_found_msg)
label = &#34;Folder&#34; if is_folder else &#34;Node&#34;
raise NodeNotFoundError(f&#34;{label} &#39;{unique_id}&#39; not found.&#34;)
@staticmethod
def _validate_node_payload(data):
if not isinstance(data, dict):
return
from ..auth_utils import (
SanitizedAuthError,
is_compose_expression,
validate_auth_budget_tags,
validate_compose_syntax,
validate_fallback_entries,
)
try:
if &#34;tags&#34; in data and data[&#34;tags&#34;] not in (&#34;&#34;, None):
validate_auth_budget_tags(data[&#34;tags&#34;])
if &#34;fallback&#34; in data and data[&#34;fallback&#34;] not in (None, &#34;&#34;, []):
validate_fallback_entries(data[&#34;fallback&#34;])
pw = data.get(&#34;password&#34;)
if is_compose_expression(pw):
validate_compose_syntax(pw)
elif isinstance(pw, (list, tuple)):
for item in pw:
if is_compose_expression(item):
validate_compose_syntax(item)
except (ValueError, TypeError, SanitizedAuthError) as exc:
raise InvalidConfigurationError(str(exc)) from None
def list_nodes(self, filter_str=None, format_str=None):
@@ -69,6 +209,10 @@ el.replaceWith(d);
flags = re.IGNORECASE if not case_sensitive else 0
nodes = [n for n in nodes if re.search(filter_str, n, flags)]
ctx_svc = self._get_context_service()
if ctx_svc is not None and not ctx_svc.is_default_all_context():
nodes = ctx_svc.filter_node_list(result=nodes)
if not format_str:
return nodes
@@ -125,17 +269,21 @@ el.replaceWith(d);
else:
flags = re.IGNORECASE if not case_sensitive else 0
folders = [f for f in folders if re.search(filter_str, f, flags)]
ctx_svc = self._get_context_service()
if ctx_svc is not None and not ctx_svc.is_default_all_context():
folders = ctx_svc.filter_node_list(result=folders)
return folders
def get_node_details(self, unique_id):
&#34;&#34;&#34;Return full configuration dictionary for a specific node.&#34;&#34;&#34;
self._ensure_in_active_context(unique_id, is_folder=False)
try:
details = self.config.getitem(unique_id)
if not details:
raise NodeNotFoundError(f&#34;Node &#39;{unique_id}&#39; not found.&#34;)
return details
except (KeyError, TypeError):
raise NodeNotFoundError(f&#34;Node &#39;{unique_id}&#39; not found.&#34;)
raise NodeNotFoundError(f&#34;Node &#39;{unique_id}&#39; not found.&#34;) from None
def explode_unique(self, unique_id):
&#34;&#34;&#34;Explode a unique ID into a dictionary of its parts.&#34;&#34;&#34;
@@ -146,25 +294,54 @@ el.replaceWith(d);
self.config._generate_nodes_cache(nodes=nodes, folders=folders, profiles=profiles)
def validate_parent_folder(self, unique_id, is_folder=False):
&#34;&#34;&#34;Check if parent folder exists for a given node unique ID.&#34;&#34;&#34;
if is_folder:
uniques = self.config._explode_unique(unique_id)
if uniques and &#34;subfolder&#34; in uniques and &#34;folder&#34; in uniques:
parent_folder = f&#34;@{uniques[&#39;folder&#39;]}&#34;
if parent_folder not in self.config._getallfolders():
&#34;&#34;&#34;Check if unique_id is in active context and its parent folder exists.&#34;&#34;&#34;
parent_folder = self._derive_parent_folder(self.config, unique_id, is_folder=is_folder)
ctx_svc = self._get_context_service()
if ctx_svc is not None:
ctx_svc.validate_in_active_context(unique_id, is_folder=is_folder)
if parent_folder is not None and not self._folder_exists(
parent_folder, self._get_raw_folders(), self.config
):
raise NodeNotFoundError(f&#34;Folder &#39;{parent_folder}&#39; not found.&#34;)
else:
node_folder = unique_id.partition(&#34;@&#34;)[2]
if node_folder:
parent_folder = f&#34;@{node_folder}&#34;
if parent_folder not in self.config._getallfolders():
raise NodeNotFoundError(f&#34;Folder &#39;{parent_folder}&#39; not found.&#34;)
def add_node(self, unique_id, data, is_folder=False):
&#34;&#34;&#34;Logic for adding a new node or folder to configuration.&#34;&#34;&#34;
if not is_folder:
self._validate_node_name(unique_id)
self._validate_node_payload(data)
self.validate_parent_folder(unique_id, is_folder=is_folder)
if _supports_transactional_update(self.config):
def _mut(cfg):
all_nodes = cfg._getallnodes_unlocked()
all_folders = cfg._getallfolders_unlocked()
parent_folder = self._derive_parent_folder(cfg, unique_id, is_folder=is_folder)
ctx_svc = self._get_context_service(cfg)
if ctx_svc is not None:
ctx_svc.validate_in_active_context(unique_id, is_folder=is_folder)
if parent_folder is not None and not self._folder_exists(
parent_folder, all_folders, cfg
):
raise NodeNotFoundError(f&#34;Folder &#39;{parent_folder}&#39; not found.&#34;)
if is_folder:
if unique_id in all_folders:
raise NodeAlreadyExistsError(f&#34;Folder &#39;{unique_id}&#39; already exists.&#34;)
uniques = cfg._explode_unique_unlocked(unique_id)
if not uniques:
raise InvalidConfigurationError(f&#34;Invalid folder name &#39;{unique_id}&#39;.&#34;)
cfg._folder_add_unlocked(**uniques)
else:
if unique_id in all_nodes:
raise NodeAlreadyExistsError(f&#34;Node &#39;{unique_id}&#39; already exists.&#34;)
payload = deepcopy(data)
if &#34;id&#34; not in payload:
uniques = cfg._explode_unique_unlocked(unique_id)
if uniques and &#34;id&#34; in uniques:
payload[&#34;id&#34;] = uniques[&#34;id&#34;]
cfg._connections_add_unlocked(**payload)
self.config.transactional_update(_mut)
return
all_nodes = self.config._getallnodes()
all_folders = self.config._getallfolders()
@@ -176,7 +353,6 @@ el.replaceWith(d);
if not uniques:
raise InvalidConfigurationError(f&#34;Invalid folder name &#39;{unique_id}&#39;.&#34;)
# Check if parent folder exists when creating a subfolder
if &#34;subfolder&#34; in uniques:
self.validate_parent_folder(unique_id, is_folder=True)
@@ -186,37 +362,75 @@ el.replaceWith(d);
if unique_id in all_nodes:
raise NodeAlreadyExistsError(f&#34;Node &#39;{unique_id}&#39; already exists.&#34;)
# Check if parent folder exists when creating a node in a folder
self.validate_parent_folder(unique_id)
# Ensure &#39;id&#39; is in data for config._connections_add
if &#34;id&#34; not in data:
payload = deepcopy(data)
if &#34;id&#34; not in payload:
uniques = self.config._explode_unique(unique_id)
if uniques and &#34;id&#34; in uniques:
data[&#34;id&#34;] = uniques[&#34;id&#34;]
payload[&#34;id&#34;] = uniques[&#34;id&#34;]
self.config._connections_add(**data)
self.config._connections_add(**payload)
self.config._saveconfig(self.config.file)
def _build_update_payload(self, unique_id, data, existing, uniques):
if isinstance(existing, dict):
merged = deepcopy(existing)
if isinstance(uniques, dict):
merged.update({k: v for k, v in uniques.items() if v})
merged.update(deepcopy(data))
filtered = {k: v for k, v in merged.items() if k in self._CONNECTION_FIELDS}
else:
filtered = deepcopy(data)
if &#34;id&#34; not in filtered and isinstance(uniques, dict) and &#34;id&#34; in uniques:
filtered[&#34;id&#34;] = uniques[&#34;id&#34;]
self._validate_node_payload(filtered)
return filtered
def update_node(self, unique_id, data, save=True):
&#34;&#34;&#34;Explicitly update an existing node.&#34;&#34;&#34;
self._ensure_in_active_context(unique_id, is_folder=False)
if _supports_transactional_update(self.config) and save:
def _mut(cfg):
all_nodes = cfg._getallnodes_unlocked()
if unique_id not in all_nodes:
raise NodeNotFoundError(f&#34;Node &#39;{unique_id}&#39; not found.&#34;)
uniques = cfg._explode_unique_unlocked(unique_id)
existing = cfg.getitem_unlocked(unique_id, extract=False)
payload = self._build_update_payload(unique_id, data, existing, uniques)
cfg._connections_add_unlocked(**payload)
self.config.transactional_update(_mut)
return
all_nodes = self.config._getallnodes()
if unique_id not in all_nodes:
raise NodeNotFoundError(f&#34;Node &#39;{unique_id}&#39; not found.&#34;)
# Ensure &#39;id&#39; is in data for config._connections_add
if &#34;id&#34; not in data:
uniques = self.config._explode_unique(unique_id)
if uniques:
data[&#34;id&#34;] = uniques[&#34;id&#34;]
# config._connections_add actually handles updates if ID exists correctly
self.config._connections_add(**data)
existing = self.config.getitem(unique_id, extract=False)
payload = self._build_update_payload(unique_id, data, existing, uniques)
self.config._connections_add(**payload)
if save:
self.config._saveconfig(self.config.file)
def delete_node(self, unique_id, is_folder=False, save=True):
&#34;&#34;&#34;Logic for deleting a node or folder.&#34;&#34;&#34;
self._ensure_in_active_context(unique_id, is_folder=is_folder)
if _supports_transactional_update(self.config) and save:
def _mut(cfg):
uniques = cfg._explode_unique_unlocked(unique_id)
if not uniques:
label = &#34;Folder&#34; if is_folder else &#34;Node&#34;
raise NodeNotFoundError(f&#34;{label} &#39;{unique_id}&#39; not found or invalid.&#34;)
if is_folder:
cfg._folder_del_unlocked(**uniques)
else:
cfg._connections_del_unlocked(**uniques)
self.config.transactional_update(_mut)
return
if is_folder:
uniques = self.config._explode_unique(unique_id)
if not uniques:
@@ -233,17 +447,35 @@ el.replaceWith(d);
def connect_node(self, unique_id, sftp=False, debug=False, logger=None):
&#34;&#34;&#34;Interact with a node directly.&#34;&#34;&#34;
self._ensure_in_active_context(unique_id, is_folder=False)
from connpy.core import node
from ..auth_utils import resolve_fallback_entries
from .profile_service import ProfileService
node_data = self.config.getitem(unique_id, extract=False)
if not node_data:
raise NodeNotFoundError(f&#34;Node &#39;{unique_id}&#39; not found.&#34;)
# Resolve profiles
profile_service = ProfileService(self.config)
resolved_data = profile_service.resolve_node_data(node_data)
node_params = inspect.signature(node.__init__).parameters
if &#34;fallback&#34; not in node_params and &#34;fallback&#34; in resolved_data:
raw_fb = resolved_data.pop(&#34;fallback&#34;, None)
if raw_fb not in (None, &#34;&#34;, []):
resolved_fb = resolve_fallback_entries(raw_fb, self.config.profiles)
if resolved_fb:
tags_val = resolved_data.get(&#34;tags&#34;, &#34;&#34;)
if isinstance(tags_val, dict):
tags_copy = deepcopy(tags_val)
tags_copy[&#34;_fallback&#34;] = resolved_fb
resolved_data[&#34;tags&#34;] = tags_copy
elif tags_val in (&#34;&#34;, None):
resolved_data[&#34;tags&#34;] = {&#34;_fallback&#34;: resolved_fb}
else:
resolved_data[&#34;tags&#34;] = {&#34;_raw&#34;: tags_val, &#34;_fallback&#34;: resolved_fb}
n = node(unique_id, **resolved_data, config=self.config)
if sftp:
n.protocol = &#34;sftp&#34;
@@ -252,7 +484,48 @@ el.replaceWith(d);
def move_node(self, src_id, dst_id, copy=False):
&#34;&#34;&#34;Move or copy a node.&#34;&#34;&#34;
self._ensure_in_active_context(
src_id, is_folder=False, not_found_msg=f&#34;Source node &#39;{src_id}&#39; not found.&#34;
)
self._validate_node_name(dst_id)
self.validate_parent_folder(dst_id, is_folder=False)
if _supports_transactional_update(self.config):
def _mut(cfg):
ctx_svc = self._get_context_service(cfg)
if (
ctx_svc is not None
and not ctx_svc.is_default_all_context()
and not ctx_svc.matches_active_context(src_id)
):
raise NodeNotFoundError(f&#34;Source node &#39;{src_id}&#39; not found.&#34;)
parent_folder = self._derive_parent_folder(cfg, dst_id, is_folder=False)
if ctx_svc is not None:
ctx_svc.validate_in_active_context(dst_id, is_folder=False)
if parent_folder is not None and not self._folder_exists(
parent_folder, cfg._getallfolders_unlocked(), cfg
):
raise NodeNotFoundError(f&#34;Folder &#39;{parent_folder}&#39; not found.&#34;)
try:
node_data = cfg.getitem_unlocked(src_id)
except (KeyError, TypeError):
node_data = None
if not node_data:
raise NodeNotFoundError(f&#34;Source node &#39;{src_id}&#39; not found.&#34;)
if dst_id in cfg._getallnodes_unlocked():
raise NodeAlreadyExistsError(f&#34;Destination node &#39;{dst_id}&#39; already exists.&#34;)
new_uniques = cfg._explode_unique_unlocked(dst_id)
if not new_uniques:
raise InvalidConfigurationError(f&#34;Invalid destination format &#39;{dst_id}&#39;.&#34;)
new_node_data = deepcopy(node_data)
new_node_data.update(new_uniques)
cfg._connections_add_unlocked(**new_node_data)
if not copy:
src_uniques = cfg._explode_unique_unlocked(src_id)
cfg._connections_del_unlocked(**src_uniques)
self.config.transactional_update(_mut)
return
node_data = self.config.getitem(src_id)
if not node_data:
@@ -278,6 +551,55 @@ el.replaceWith(d);
def bulk_add(self, ids, hosts, common_data):
&#34;&#34;&#34;Add multiple nodes with shared common configuration.&#34;&#34;&#34;
self._validate_node_payload(common_data)
for uid in ids:
self.validate_parent_folder(uid, is_folder=False)
if _supports_transactional_update(self.config):
def _build_entries(cfg):
all_nodes = cfg._getallnodes_unlocked()
all_folders = cfg._getallfolders_unlocked()
ctx_svc = self._get_context_service(cfg)
planned = []
for i, uid in enumerate(ids):
parent_folder = self._derive_parent_folder(cfg, uid, is_folder=False)
if ctx_svc is not None:
ctx_svc.validate_in_active_context(uid, is_folder=False)
if parent_folder is not None and not self._folder_exists(
parent_folder, all_folders, cfg
):
raise NodeNotFoundError(f&#34;Folder &#39;{parent_folder}&#39; not found.&#34;)
if uid in all_nodes:
continue
try:
self._validate_node_name(uid)
except ReservedNameError:
continue
host = hosts[i] if i &lt; len(hosts) else hosts[0]
uniques = cfg._explode_unique_unlocked(uid)
if not uniques:
continue
node_data = deepcopy(common_data)
node_data.pop(&#34;ids&#34;, None)
node_data.pop(&#34;location&#34;, None)
node_data.update(uniques)
node_data[&#34;host&#34;] = host
node_data[&#34;type&#34;] = &#34;connection&#34;
planned.append(node_data)
return planned
with self.config._mutation_lock:
if not _build_entries(self.config):
return 0
def _mut(cfg):
entries = _build_entries(cfg)
for node_data in entries:
cfg._connections_add_unlocked(**node_data)
return len(entries)
return self.config.transactional_update(_mut)
count = 0
all_nodes = self.config._getallnodes()
@@ -288,8 +610,6 @@ el.replaceWith(d);
try:
self._validate_node_name(uid)
except ReservedNameError:
# For bulk, we might want to just skip or log.
# CLI caller will handle if it wants to be strict.
continue
host = hosts[i] if i &lt; len(hosts) else hosts[0]
@@ -313,6 +633,21 @@ el.replaceWith(d);
def full_replace(self, connections, profiles):
&#34;&#34;&#34;Replace all connections and profiles with new data.&#34;&#34;&#34;
from ..auth_utils import SanitizedAuthError
if _supports_transactional_update(self.config):
def _mut(cfg):
cfg.connections.clear()
cfg.connections.update(deepcopy(connections))
cfg.profiles.clear()
cfg.profiles.update(deepcopy(profiles))
try:
self.config.transactional_update(_mut)
except (ValueError, TypeError, SanitizedAuthError) as exc:
raise InvalidConfigurationError(str(exc)) from None
return
self.config.connections = connections
self.config.profiles = profiles
self.config._saveconfig(self.config.file)
@@ -326,8 +661,9 @@ el.replaceWith(d);
</details>
<div class="desc"><p>Base class for all connpy services, providing common configuration access.</p>
<p>Initialize the service.</p>
<h2 id="args">Args</h2>
<dl>
<dt>Args</dt>
<dt>-----=</dt>
<dt><strong><code>config</code></strong></dt>
<dd>An instance of configfile (or None to instantiate a new one/use global context).</dd>
</dl></div>
@@ -349,6 +685,40 @@ el.replaceWith(d);
&#34;&#34;&#34;Logic for adding a new node or folder to configuration.&#34;&#34;&#34;
if not is_folder:
self._validate_node_name(unique_id)
self._validate_node_payload(data)
self.validate_parent_folder(unique_id, is_folder=is_folder)
if _supports_transactional_update(self.config):
def _mut(cfg):
all_nodes = cfg._getallnodes_unlocked()
all_folders = cfg._getallfolders_unlocked()
parent_folder = self._derive_parent_folder(cfg, unique_id, is_folder=is_folder)
ctx_svc = self._get_context_service(cfg)
if ctx_svc is not None:
ctx_svc.validate_in_active_context(unique_id, is_folder=is_folder)
if parent_folder is not None and not self._folder_exists(
parent_folder, all_folders, cfg
):
raise NodeNotFoundError(f&#34;Folder &#39;{parent_folder}&#39; not found.&#34;)
if is_folder:
if unique_id in all_folders:
raise NodeAlreadyExistsError(f&#34;Folder &#39;{unique_id}&#39; already exists.&#34;)
uniques = cfg._explode_unique_unlocked(unique_id)
if not uniques:
raise InvalidConfigurationError(f&#34;Invalid folder name &#39;{unique_id}&#39;.&#34;)
cfg._folder_add_unlocked(**uniques)
else:
if unique_id in all_nodes:
raise NodeAlreadyExistsError(f&#34;Node &#39;{unique_id}&#39; already exists.&#34;)
payload = deepcopy(data)
if &#34;id&#34; not in payload:
uniques = cfg._explode_unique_unlocked(unique_id)
if uniques and &#34;id&#34; in uniques:
payload[&#34;id&#34;] = uniques[&#34;id&#34;]
cfg._connections_add_unlocked(**payload)
self.config.transactional_update(_mut)
return
all_nodes = self.config._getallnodes()
all_folders = self.config._getallfolders()
@@ -360,7 +730,6 @@ el.replaceWith(d);
if not uniques:
raise InvalidConfigurationError(f&#34;Invalid folder name &#39;{unique_id}&#39;.&#34;)
# Check if parent folder exists when creating a subfolder
if &#34;subfolder&#34; in uniques:
self.validate_parent_folder(unique_id, is_folder=True)
@@ -370,16 +739,15 @@ el.replaceWith(d);
if unique_id in all_nodes:
raise NodeAlreadyExistsError(f&#34;Node &#39;{unique_id}&#39; already exists.&#34;)
# Check if parent folder exists when creating a node in a folder
self.validate_parent_folder(unique_id)
# Ensure &#39;id&#39; is in data for config._connections_add
if &#34;id&#34; not in data:
payload = deepcopy(data)
if &#34;id&#34; not in payload:
uniques = self.config._explode_unique(unique_id)
if uniques and &#34;id&#34; in uniques:
data[&#34;id&#34;] = uniques[&#34;id&#34;]
payload[&#34;id&#34;] = uniques[&#34;id&#34;]
self.config._connections_add(**data)
self.config._connections_add(**payload)
self.config._saveconfig(self.config.file)</code></pre>
</details>
<div class="desc"><p>Logic for adding a new node or folder to configuration.</p></div>
@@ -394,6 +762,55 @@ el.replaceWith(d);
</summary>
<pre><code class="python">def bulk_add(self, ids, hosts, common_data):
&#34;&#34;&#34;Add multiple nodes with shared common configuration.&#34;&#34;&#34;
self._validate_node_payload(common_data)
for uid in ids:
self.validate_parent_folder(uid, is_folder=False)
if _supports_transactional_update(self.config):
def _build_entries(cfg):
all_nodes = cfg._getallnodes_unlocked()
all_folders = cfg._getallfolders_unlocked()
ctx_svc = self._get_context_service(cfg)
planned = []
for i, uid in enumerate(ids):
parent_folder = self._derive_parent_folder(cfg, uid, is_folder=False)
if ctx_svc is not None:
ctx_svc.validate_in_active_context(uid, is_folder=False)
if parent_folder is not None and not self._folder_exists(
parent_folder, all_folders, cfg
):
raise NodeNotFoundError(f&#34;Folder &#39;{parent_folder}&#39; not found.&#34;)
if uid in all_nodes:
continue
try:
self._validate_node_name(uid)
except ReservedNameError:
continue
host = hosts[i] if i &lt; len(hosts) else hosts[0]
uniques = cfg._explode_unique_unlocked(uid)
if not uniques:
continue
node_data = deepcopy(common_data)
node_data.pop(&#34;ids&#34;, None)
node_data.pop(&#34;location&#34;, None)
node_data.update(uniques)
node_data[&#34;host&#34;] = host
node_data[&#34;type&#34;] = &#34;connection&#34;
planned.append(node_data)
return planned
with self.config._mutation_lock:
if not _build_entries(self.config):
return 0
def _mut(cfg):
entries = _build_entries(cfg)
for node_data in entries:
cfg._connections_add_unlocked(**node_data)
return len(entries)
return self.config.transactional_update(_mut)
count = 0
all_nodes = self.config._getallnodes()
@@ -404,8 +821,6 @@ el.replaceWith(d);
try:
self._validate_node_name(uid)
except ReservedNameError:
# For bulk, we might want to just skip or log.
# CLI caller will handle if it wants to be strict.
continue
host = hosts[i] if i &lt; len(hosts) else hosts[0]
@@ -439,17 +854,35 @@ el.replaceWith(d);
</summary>
<pre><code class="python">def connect_node(self, unique_id, sftp=False, debug=False, logger=None):
&#34;&#34;&#34;Interact with a node directly.&#34;&#34;&#34;
self._ensure_in_active_context(unique_id, is_folder=False)
from connpy.core import node
from ..auth_utils import resolve_fallback_entries
from .profile_service import ProfileService
node_data = self.config.getitem(unique_id, extract=False)
if not node_data:
raise NodeNotFoundError(f&#34;Node &#39;{unique_id}&#39; not found.&#34;)
# Resolve profiles
profile_service = ProfileService(self.config)
resolved_data = profile_service.resolve_node_data(node_data)
node_params = inspect.signature(node.__init__).parameters
if &#34;fallback&#34; not in node_params and &#34;fallback&#34; in resolved_data:
raw_fb = resolved_data.pop(&#34;fallback&#34;, None)
if raw_fb not in (None, &#34;&#34;, []):
resolved_fb = resolve_fallback_entries(raw_fb, self.config.profiles)
if resolved_fb:
tags_val = resolved_data.get(&#34;tags&#34;, &#34;&#34;)
if isinstance(tags_val, dict):
tags_copy = deepcopy(tags_val)
tags_copy[&#34;_fallback&#34;] = resolved_fb
resolved_data[&#34;tags&#34;] = tags_copy
elif tags_val in (&#34;&#34;, None):
resolved_data[&#34;tags&#34;] = {&#34;_fallback&#34;: resolved_fb}
else:
resolved_data[&#34;tags&#34;] = {&#34;_raw&#34;: tags_val, &#34;_fallback&#34;: resolved_fb}
n = node(unique_id, **resolved_data, config=self.config)
if sftp:
n.protocol = &#34;sftp&#34;
@@ -468,6 +901,21 @@ el.replaceWith(d);
</summary>
<pre><code class="python">def delete_node(self, unique_id, is_folder=False, save=True):
&#34;&#34;&#34;Logic for deleting a node or folder.&#34;&#34;&#34;
self._ensure_in_active_context(unique_id, is_folder=is_folder)
if _supports_transactional_update(self.config) and save:
def _mut(cfg):
uniques = cfg._explode_unique_unlocked(unique_id)
if not uniques:
label = &#34;Folder&#34; if is_folder else &#34;Node&#34;
raise NodeNotFoundError(f&#34;{label} &#39;{unique_id}&#39; not found or invalid.&#34;)
if is_folder:
cfg._folder_del_unlocked(**uniques)
else:
cfg._connections_del_unlocked(**uniques)
self.config.transactional_update(_mut)
return
if is_folder:
uniques = self.config._explode_unique(unique_id)
if not uniques:
@@ -508,6 +956,21 @@ el.replaceWith(d);
</summary>
<pre><code class="python">def full_replace(self, connections, profiles):
&#34;&#34;&#34;Replace all connections and profiles with new data.&#34;&#34;&#34;
from ..auth_utils import SanitizedAuthError
if _supports_transactional_update(self.config):
def _mut(cfg):
cfg.connections.clear()
cfg.connections.update(deepcopy(connections))
cfg.profiles.clear()
cfg.profiles.update(deepcopy(profiles))
try:
self.config.transactional_update(_mut)
except (ValueError, TypeError, SanitizedAuthError) as exc:
raise InvalidConfigurationError(str(exc)) from None
return
self.config.connections = connections
self.config.profiles = profiles
self.config._saveconfig(self.config.file)</code></pre>
@@ -555,13 +1018,14 @@ el.replaceWith(d);
</summary>
<pre><code class="python">def get_node_details(self, unique_id):
&#34;&#34;&#34;Return full configuration dictionary for a specific node.&#34;&#34;&#34;
self._ensure_in_active_context(unique_id, is_folder=False)
try:
details = self.config.getitem(unique_id)
if not details:
raise NodeNotFoundError(f&#34;Node &#39;{unique_id}&#39; not found.&#34;)
return details
except (KeyError, TypeError):
raise NodeNotFoundError(f&#34;Node &#39;{unique_id}&#39; not found.&#34;)</code></pre>
raise NodeNotFoundError(f&#34;Node &#39;{unique_id}&#39; not found.&#34;) from None</code></pre>
</details>
<div class="desc"><p>Return full configuration dictionary for a specific node.</p></div>
</dd>
@@ -587,6 +1051,9 @@ el.replaceWith(d);
else:
flags = re.IGNORECASE if not case_sensitive else 0
folders = [f for f in folders if re.search(filter_str, f, flags)]
ctx_svc = self._get_context_service()
if ctx_svc is not None and not ctx_svc.is_default_all_context():
folders = ctx_svc.filter_node_list(result=folders)
return folders</code></pre>
</details>
<div class="desc"><p>Return all unique folders, optionally filtered by regex.</p></div>
@@ -608,6 +1075,10 @@ el.replaceWith(d);
flags = re.IGNORECASE if not case_sensitive else 0
nodes = [n for n in nodes if re.search(filter_str, n, flags)]
ctx_svc = self._get_context_service()
if ctx_svc is not None and not ctx_svc.is_default_all_context():
nodes = ctx_svc.filter_node_list(result=nodes)
if not format_str:
return nodes
@@ -662,7 +1133,48 @@ el.replaceWith(d);
</summary>
<pre><code class="python">def move_node(self, src_id, dst_id, copy=False):
&#34;&#34;&#34;Move or copy a node.&#34;&#34;&#34;
self._ensure_in_active_context(
src_id, is_folder=False, not_found_msg=f&#34;Source node &#39;{src_id}&#39; not found.&#34;
)
self._validate_node_name(dst_id)
self.validate_parent_folder(dst_id, is_folder=False)
if _supports_transactional_update(self.config):
def _mut(cfg):
ctx_svc = self._get_context_service(cfg)
if (
ctx_svc is not None
and not ctx_svc.is_default_all_context()
and not ctx_svc.matches_active_context(src_id)
):
raise NodeNotFoundError(f&#34;Source node &#39;{src_id}&#39; not found.&#34;)
parent_folder = self._derive_parent_folder(cfg, dst_id, is_folder=False)
if ctx_svc is not None:
ctx_svc.validate_in_active_context(dst_id, is_folder=False)
if parent_folder is not None and not self._folder_exists(
parent_folder, cfg._getallfolders_unlocked(), cfg
):
raise NodeNotFoundError(f&#34;Folder &#39;{parent_folder}&#39; not found.&#34;)
try:
node_data = cfg.getitem_unlocked(src_id)
except (KeyError, TypeError):
node_data = None
if not node_data:
raise NodeNotFoundError(f&#34;Source node &#39;{src_id}&#39; not found.&#34;)
if dst_id in cfg._getallnodes_unlocked():
raise NodeAlreadyExistsError(f&#34;Destination node &#39;{dst_id}&#39; already exists.&#34;)
new_uniques = cfg._explode_unique_unlocked(dst_id)
if not new_uniques:
raise InvalidConfigurationError(f&#34;Invalid destination format &#39;{dst_id}&#39;.&#34;)
new_node_data = deepcopy(node_data)
new_node_data.update(new_uniques)
cfg._connections_add_unlocked(**new_node_data)
if not copy:
src_uniques = cfg._explode_unique_unlocked(src_id)
cfg._connections_del_unlocked(**src_uniques)
self.config.transactional_update(_mut)
return
node_data = self.config.getitem(src_id)
if not node_data:
@@ -698,18 +1210,28 @@ el.replaceWith(d);
</summary>
<pre><code class="python">def update_node(self, unique_id, data, save=True):
&#34;&#34;&#34;Explicitly update an existing node.&#34;&#34;&#34;
self._ensure_in_active_context(unique_id, is_folder=False)
if _supports_transactional_update(self.config) and save:
def _mut(cfg):
all_nodes = cfg._getallnodes_unlocked()
if unique_id not in all_nodes:
raise NodeNotFoundError(f&#34;Node &#39;{unique_id}&#39; not found.&#34;)
uniques = cfg._explode_unique_unlocked(unique_id)
existing = cfg.getitem_unlocked(unique_id, extract=False)
payload = self._build_update_payload(unique_id, data, existing, uniques)
cfg._connections_add_unlocked(**payload)
self.config.transactional_update(_mut)
return
all_nodes = self.config._getallnodes()
if unique_id not in all_nodes:
raise NodeNotFoundError(f&#34;Node &#39;{unique_id}&#39; not found.&#34;)
# Ensure &#39;id&#39; is in data for config._connections_add
if &#34;id&#34; not in data:
uniques = self.config._explode_unique(unique_id)
if uniques:
data[&#34;id&#34;] = uniques[&#34;id&#34;]
# config._connections_add actually handles updates if ID exists correctly
self.config._connections_add(**data)
existing = self.config.getitem(unique_id, extract=False)
payload = self._build_update_payload(unique_id, data, existing, uniques)
self.config._connections_add(**payload)
if save:
self.config._saveconfig(self.config.file)</code></pre>
</details>
@@ -724,21 +1246,17 @@ el.replaceWith(d);
<span>Expand source code</span>
</summary>
<pre><code class="python">def validate_parent_folder(self, unique_id, is_folder=False):
&#34;&#34;&#34;Check if parent folder exists for a given node unique ID.&#34;&#34;&#34;
if is_folder:
uniques = self.config._explode_unique(unique_id)
if uniques and &#34;subfolder&#34; in uniques and &#34;folder&#34; in uniques:
parent_folder = f&#34;@{uniques[&#39;folder&#39;]}&#34;
if parent_folder not in self.config._getallfolders():
raise NodeNotFoundError(f&#34;Folder &#39;{parent_folder}&#39; not found.&#34;)
else:
node_folder = unique_id.partition(&#34;@&#34;)[2]
if node_folder:
parent_folder = f&#34;@{node_folder}&#34;
if parent_folder not in self.config._getallfolders():
&#34;&#34;&#34;Check if unique_id is in active context and its parent folder exists.&#34;&#34;&#34;
parent_folder = self._derive_parent_folder(self.config, unique_id, is_folder=is_folder)
ctx_svc = self._get_context_service()
if ctx_svc is not None:
ctx_svc.validate_in_active_context(unique_id, is_folder=is_folder)
if parent_folder is not None and not self._folder_exists(
parent_folder, self._get_raw_folders(), self.config
):
raise NodeNotFoundError(f&#34;Folder &#39;{parent_folder}&#39; not found.&#34;)</code></pre>
</details>
<div class="desc"><p>Check if parent folder exists for a given node unique ID.</p></div>
<div class="desc"><p>Check if unique_id is in active context and its parent folder exists.</p></div>
</dd>
</dl>
<h3>Inherited members</h3>
+2 -1
View File
@@ -411,8 +411,9 @@ el.replaceWith(d);
</details>
<div class="desc"><p>Business logic for enabling, disabling, and listing plugins.</p>
<p>Initialize the service.</p>
<h2 id="args">Args</h2>
<dl>
<dt>Args</dt>
<dt>-----=</dt>
<dt><strong><code>config</code></strong></dt>
<dd>An instance of configfile (or None to instantiate a new one/use global context).</dd>
</dl></div>
+366 -114
View File
@@ -58,61 +58,189 @@ el.replaceWith(d);
<pre><code class="python">class ProfileService(BaseService):
&#34;&#34;&#34;Business logic for node profiles management.&#34;&#34;&#34;
_ALLOWED_KEYS: ClassVar[frozenset[str]] = frozenset({
&#34;host&#34;,
&#34;options&#34;,
&#34;logs&#34;,
&#34;password&#34;,
&#34;port&#34;,
&#34;protocol&#34;,
&#34;user&#34;,
&#34;tags&#34;,
&#34;jumphost&#34;,
&#34;fallback&#34;,
})
@staticmethod
def _validate_profile_payload(data):
from ..auth_utils import (
SanitizedAuthError,
is_compose_expression,
validate_auth_budget_tags,
validate_compose_syntax,
validate_fallback_entries,
)
try:
if &#34;tags&#34; in data and data[&#34;tags&#34;] not in (&#34;&#34;, None):
validate_auth_budget_tags(data[&#34;tags&#34;])
if &#34;fallback&#34; in data and data[&#34;fallback&#34;] not in (None, &#34;&#34;, []):
validate_fallback_entries(data[&#34;fallback&#34;])
pw = data.get(&#34;password&#34;)
if is_compose_expression(pw):
validate_compose_syntax(pw)
elif isinstance(pw, (list, tuple)):
for item in pw:
if is_compose_expression(item):
validate_compose_syntax(item)
except (ValueError, TypeError, SanitizedAuthError) as exc:
raise InvalidConfigurationError(str(exc)) from None
def _get_profiles_snapshot(self):
lock = getattr(self.config, &#34;_mutation_lock&#34;, None)
if lock is not None and not hasattr(self.config, &#34;assert_called&#34;):
with lock:
return deepcopy(self.config.profiles)
return deepcopy(self.config.profiles)
def list_profiles(self, filter_str=None):
&#34;&#34;&#34;List all profile names, optionally filtered.&#34;&#34;&#34;
profiles = list(self.config.profiles.keys())
profiles = list(self._get_profiles_snapshot().keys())
case_sensitive = self.config.config.get(&#34;case&#34;, False)
if filter_str:
if not case_sensitive:
f_str = filter_str.lower()
return [p for p in profiles if f_str in p.lower()]
else:
return [p for p in profiles if filter_str in p]
return profiles
def get_profile(self, name, resolve=True):
def get_profile(self, name, resolve=True, *, _visited=frozenset(), _depth=0):
&#34;&#34;&#34;Get the profile dictionary, optionally resolved.&#34;&#34;&#34;
profile = self.config.profiles.get(name)
if _depth &gt;= MAX_PROFILE_DEPTH or name in _visited:
raise InvalidConfigurationError(
f&#34;Circular or overly deep profile reference involving &#39;{name}&#39;.&#34;
)
profiles = self._get_profiles_snapshot()
profile = profiles.get(name)
if not profile:
raise ProfileNotFoundError(f&#34;Profile &#39;{name}&#39; not found.&#34;)
if resolve:
return self.resolve_node_data(profile)
return self.resolve_node_data(
profile,
_visited=_visited | {name},
_depth=_depth + 1,
_profiles_snapshot=profiles,
)
return profile
def add_profile(self, name, data):
&#34;&#34;&#34;Add a new profile.&#34;&#34;&#34;
filtered_data = {k: deepcopy(v) for k, v in data.items() if k in self._ALLOWED_KEYS}
self._validate_profile_payload(filtered_data)
if _supports_transactional_update(self.config):
def _mut(cfg):
if name in cfg.profiles:
raise ProfileAlreadyExistsError(f&#34;Profile &#39;{name}&#39; already exists.&#34;)
cfg._profiles_add_unlocked(id=name, **filtered_data)
self.config.transactional_update(_mut)
return
if name in self.config.profiles:
raise ProfileAlreadyExistsError(f&#34;Profile &#39;{name}&#39; already exists.&#34;)
# Filter data to match _profiles_add signature and ensure id is passed
allowed_keys = {&#34;host&#34;, &#34;options&#34;, &#34;logs&#34;, &#34;password&#34;, &#34;port&#34;, &#34;protocol&#34;, &#34;user&#34;, &#34;tags&#34;, &#34;jumphost&#34;}
filtered_data = {k: v for k, v in data.items() if k in allowed_keys}
self.config._profiles_add(id=name, **filtered_data)
self.config._saveconfig(self.config.file)
def resolve_node_data(self, node_data):
&#34;&#34;&#34;Resolve profile references (@profile) in node data and handle inheritance.&#34;&#34;&#34;
resolved = node_data.copy()
def resolve_node_data(
self,
node_data,
*,
_visited=frozenset(),
_depth=0,
_profiles_snapshot=None,
):
&#34;&#34;&#34;Resolve profile references (@profile) in node data while preserving compose and scalar secrets.&#34;&#34;&#34;
from ..auth_utils import is_compose_expression
# 1. Identify all referenced profiles to support inheritance
referenced_profiles = []
for value in resolved.values():
if isinstance(value, str) and value.startswith(&#34;@&#34;):
referenced_profiles.append(value[1:])
elif isinstance(value, list):
if _depth &gt; MAX_PROFILE_DEPTH:
raise InvalidConfigurationError(&#34;Profile inheritance exceeds maximum depth.&#34;)
profiles = (
_profiles_snapshot
if _profiles_snapshot is not None
else self._get_profiles_snapshot()
)
resolved = deepcopy(node_data)
auth_referenced_profiles = []
for key in (&#34;user&#34;, &#34;password&#34;):
value = resolved.get(key)
if isinstance(value, str):
if not is_compose_expression(value) and value.startswith(&#34;@&#34;):
ref = value[1:]
if key != &#34;password&#34; or ref in profiles:
auth_referenced_profiles.append(ref)
elif isinstance(value, (list, tuple)):
for item in value:
if isinstance(item, str) and item.startswith(&#34;@&#34;):
referenced_profiles.append(item[1:])
if (
isinstance(item, str)
and not is_compose_expression(item)
and item.startswith(&#34;@&#34;)
):
ref = item[1:]
if key != &#34;password&#34; or ref in profiles:
auth_referenced_profiles.append(ref)
def _get_resolved_ref(prof_name):
if prof_name in _visited or _depth &gt;= MAX_PROFILE_DEPTH:
raise InvalidConfigurationError(
f&#34;Circular or overly deep profile reference involving &#39;{prof_name}&#39;.&#34;
)
prof_raw = profiles.get(prof_name)
if not prof_raw:
raise ProfileNotFoundError(f&#34;Profile &#39;{prof_name}&#39; not found.&#34;)
return self.resolve_node_data(
prof_raw,
_visited=_visited | {prof_name},
_depth=_depth + 1,
_profiles_snapshot=profiles,
)
for key, value in list(resolved.items()):
if key == &#34;fallback&#34;:
continue
if key == &#34;password&#34;:
if isinstance(value, str):
if is_compose_expression(value):
continue
if value.startswith(&#34;@&#34;) and value[1:] in profiles:
prof_obj = _get_resolved_ref(value[1:])
resolved[&#34;password&#34;] = prof_obj.get(&#34;password&#34;, &#34;&#34;)
elif isinstance(value, list):
resolved_list = []
for item in value:
if (
isinstance(item, str)
and not is_compose_expression(item)
and item.startswith(&#34;@&#34;)
and item[1:] in profiles
):
with suppress(ProfileNotFoundError):
prof_obj = _get_resolved_ref(item[1:])
if &#34;password&#34; in prof_obj:
resolved_list.append(prof_obj[&#34;password&#34;])
else:
resolved_list.append(item)
resolved[&#34;password&#34;] = resolved_list
continue
# 2. Resolve explicit references
for key, value in resolved.items():
if isinstance(value, str) and value.startswith(&#34;@&#34;):
profile_name = value[1:]
try:
profile = self.get_profile(profile_name, resolve=True)
profile = _get_resolved_ref(profile_name)
resolved[key] = profile.get(key, &#34;&#34;)
except ProfileNotFoundError:
resolved[key] = &#34;&#34;
@@ -121,33 +249,29 @@ el.replaceWith(d);
for item in value:
if isinstance(item, str) and item.startswith(&#34;@&#34;):
profile_name = item[1:]
try:
profile = self.get_profile(profile_name, resolve=True)
with suppress(ProfileNotFoundError):
profile = _get_resolved_ref(profile_name)
if &#34;password&#34; in profile:
resolved_list.append(profile[&#34;password&#34;])
except ProfileNotFoundError:
pass
else:
resolved_list.append(item)
resolved[key] = resolved_list
# 3. Inheritance: Fill empty keys from the first referenced profile
if referenced_profiles:
base_profile_name = referenced_profiles[0]
try:
base_profile = self.get_profile(base_profile_name, resolve=True)
for key, value in base_profile.items():
# Fill if key is missing or empty
if key not in resolved or resolved[key] == &#34;&#34; or resolved[key] == [] or resolved[key] is None:
resolved[key] = value
except ProfileNotFoundError:
pass
dedup_auth_profiles = list(dict.fromkeys(auth_referenced_profiles))
for base_profile_name in dedup_auth_profiles:
with suppress(ProfileNotFoundError):
base_profile = _get_resolved_ref(base_profile_name)
for key in (&#34;user&#34;, &#34;password&#34;, &#34;fallback&#34;):
val = base_profile.get(key)
if (
key not in resolved or resolved[key] in (&#34;&#34;, [], None)
) and val not in (&#34;&#34;, [], None):
resolved[key] = deepcopy(val)
# 4. Handle default protocol
if resolved.get(&#34;protocol&#34;) == &#34;&#34; or resolved.get(&#34;protocol&#34;) is None:
if resolved.get(&#34;protocol&#34;) in (&#34;&#34;, None):
try:
default_profile = self.get_profile(&#34;default&#34;, resolve=True)
resolved[&#34;protocol&#34;] = default_profile.get(&#34;protocol&#34;, &#34;ssh&#34;)
default_profile = _get_resolved_ref(&#34;default&#34;)
resolved[&#34;protocol&#34;] = default_profile.get(&#34;protocol&#34;, &#34;ssh&#34;) or &#34;ssh&#34;
except ProfileNotFoundError:
resolved[&#34;protocol&#34;] = &#34;ssh&#34;
@@ -155,41 +279,66 @@ el.replaceWith(d);
def delete_profile(self, name):
&#34;&#34;&#34;Delete an existing profile, with safety checks.&#34;&#34;&#34;
if name not in self.config.profiles:
if _supports_transactional_update(self.config):
def _mut(cfg):
if name not in cfg.profiles:
raise ProfileNotFoundError(f&#34;Profile &#39;{name}&#39; not found.&#34;)
if name == &#34;default&#34;:
raise InvalidConfigurationError(&#34;Cannot delete the &#39;default&#39; profile.&#34;)
used_by = cfg._profileused_unlocked(name)
if used_by:
raise InvalidConfigurationError(
f&#34;Profile &#39;{name}&#39; is used by nodes: {&#39;, &#39;.join(used_by)}&#34;
)
cfg._profiles_del_unlocked(id=name)
self.config.transactional_update(_mut)
return
if name not in self.config.profiles:
raise ProfileNotFoundError(f&#34;Profile &#39;{name}&#39; not found.&#34;)
if name == &#34;default&#34;:
raise InvalidConfigurationError(&#34;Cannot delete the &#39;default&#39; profile.&#34;)
used_by = self.config._profileused(name)
if used_by:
# We return the list of nodes using it so the UI can inform the user
raise InvalidConfigurationError(f&#34;Profile &#39;{name}&#39; is used by nodes: {&#39;, &#39;.join(used_by)}&#34;)
raise InvalidConfigurationError(
f&#34;Profile &#39;{name}&#39; is used by nodes: {&#39;, &#39;.join(used_by)}&#34;
)
self.config._profiles_del(id=name)
self.config._saveconfig(self.config.file)
def update_profile(self, name, data):
&#34;&#34;&#34;Update an existing profile.&#34;&#34;&#34;
if _supports_transactional_update(self.config):
def _mut(cfg):
if name not in cfg.profiles:
raise ProfileNotFoundError(f&#34;Profile &#39;{name}&#39; not found.&#34;)
updated_data = deepcopy(cfg.profiles[name])
updated_data.update(deepcopy(data))
filtered_data = {
k: v for k, v in updated_data.items() if k in self._ALLOWED_KEYS
}
self._validate_profile_payload(filtered_data)
cfg._profiles_add_unlocked(id=name, **filtered_data)
self.config.transactional_update(_mut)
return
if name not in self.config.profiles:
raise ProfileNotFoundError(f&#34;Profile &#39;{name}&#39; not found.&#34;)
# Merge with existing data
existing = self.get_profile(name, resolve=False)
updated_data = existing.copy()
updated_data = deepcopy(existing)
updated_data.update(data)
# Filter data to match _profiles_add signature
allowed_keys = {&#34;host&#34;, &#34;options&#34;, &#34;logs&#34;, &#34;password&#34;, &#34;port&#34;, &#34;protocol&#34;, &#34;user&#34;, &#34;tags&#34;, &#34;jumphost&#34;}
filtered_data = {k: v for k, v in updated_data.items() if k in allowed_keys}
filtered_data = {k: v for k, v in updated_data.items() if k in self._ALLOWED_KEYS}
self._validate_profile_payload(filtered_data)
self.config._profiles_add(id=name, **filtered_data)
self.config._saveconfig(self.config.file)</code></pre>
</details>
<div class="desc"><p>Business logic for node profiles management.</p>
<p>Initialize the service.</p>
<h2 id="args">Args</h2>
<dl>
<dt>Args</dt>
<dt>-----=</dt>
<dt><strong><code>config</code></strong></dt>
<dd>An instance of configfile (or None to instantiate a new one/use global context).</dd>
</dl></div>
@@ -209,13 +358,20 @@ el.replaceWith(d);
</summary>
<pre><code class="python">def add_profile(self, name, data):
&#34;&#34;&#34;Add a new profile.&#34;&#34;&#34;
filtered_data = {k: deepcopy(v) for k, v in data.items() if k in self._ALLOWED_KEYS}
self._validate_profile_payload(filtered_data)
if _supports_transactional_update(self.config):
def _mut(cfg):
if name in cfg.profiles:
raise ProfileAlreadyExistsError(f&#34;Profile &#39;{name}&#39; already exists.&#34;)
cfg._profiles_add_unlocked(id=name, **filtered_data)
self.config.transactional_update(_mut)
return
if name in self.config.profiles:
raise ProfileAlreadyExistsError(f&#34;Profile &#39;{name}&#39; already exists.&#34;)
# Filter data to match _profiles_add signature and ensure id is passed
allowed_keys = {&#34;host&#34;, &#34;options&#34;, &#34;logs&#34;, &#34;password&#34;, &#34;port&#34;, &#34;protocol&#34;, &#34;user&#34;, &#34;tags&#34;, &#34;jumphost&#34;}
filtered_data = {k: v for k, v in data.items() if k in allowed_keys}
self.config._profiles_add(id=name, **filtered_data)
self.config._saveconfig(self.config.file)</code></pre>
</details>
@@ -231,17 +387,31 @@ el.replaceWith(d);
</summary>
<pre><code class="python">def delete_profile(self, name):
&#34;&#34;&#34;Delete an existing profile, with safety checks.&#34;&#34;&#34;
if name not in self.config.profiles:
if _supports_transactional_update(self.config):
def _mut(cfg):
if name not in cfg.profiles:
raise ProfileNotFoundError(f&#34;Profile &#39;{name}&#39; not found.&#34;)
if name == &#34;default&#34;:
raise InvalidConfigurationError(&#34;Cannot delete the &#39;default&#39; profile.&#34;)
used_by = cfg._profileused_unlocked(name)
if used_by:
raise InvalidConfigurationError(
f&#34;Profile &#39;{name}&#39; is used by nodes: {&#39;, &#39;.join(used_by)}&#34;
)
cfg._profiles_del_unlocked(id=name)
self.config.transactional_update(_mut)
return
if name not in self.config.profiles:
raise ProfileNotFoundError(f&#34;Profile &#39;{name}&#39; not found.&#34;)
if name == &#34;default&#34;:
raise InvalidConfigurationError(&#34;Cannot delete the &#39;default&#39; profile.&#34;)
used_by = self.config._profileused(name)
if used_by:
# We return the list of nodes using it so the UI can inform the user
raise InvalidConfigurationError(f&#34;Profile &#39;{name}&#39; is used by nodes: {&#39;, &#39;.join(used_by)}&#34;)
raise InvalidConfigurationError(
f&#34;Profile &#39;{name}&#39; is used by nodes: {&#39;, &#39;.join(used_by)}&#34;
)
self.config._profiles_del(id=name)
self.config._saveconfig(self.config.file)</code></pre>
</details>
@@ -255,14 +425,24 @@ el.replaceWith(d);
<summary>
<span>Expand source code</span>
</summary>
<pre><code class="python">def get_profile(self, name, resolve=True):
<pre><code class="python">def get_profile(self, name, resolve=True, *, _visited=frozenset(), _depth=0):
&#34;&#34;&#34;Get the profile dictionary, optionally resolved.&#34;&#34;&#34;
profile = self.config.profiles.get(name)
if _depth &gt;= MAX_PROFILE_DEPTH or name in _visited:
raise InvalidConfigurationError(
f&#34;Circular or overly deep profile reference involving &#39;{name}&#39;.&#34;
)
profiles = self._get_profiles_snapshot()
profile = profiles.get(name)
if not profile:
raise ProfileNotFoundError(f&#34;Profile &#39;{name}&#39; not found.&#34;)
if resolve:
return self.resolve_node_data(profile)
return self.resolve_node_data(
profile,
_visited=_visited | {name},
_depth=_depth + 1,
_profiles_snapshot=profiles,
)
return profile</code></pre>
</details>
<div class="desc"><p>Get the profile dictionary, optionally resolved.</p></div>
@@ -277,14 +457,13 @@ el.replaceWith(d);
</summary>
<pre><code class="python">def list_profiles(self, filter_str=None):
&#34;&#34;&#34;List all profile names, optionally filtered.&#34;&#34;&#34;
profiles = list(self.config.profiles.keys())
profiles = list(self._get_profiles_snapshot().keys())
case_sensitive = self.config.config.get(&#34;case&#34;, False)
if filter_str:
if not case_sensitive:
f_str = filter_str.lower()
return [p for p in profiles if f_str in p.lower()]
else:
return [p for p in profiles if filter_str in p]
return profiles</code></pre>
</details>
@@ -298,26 +477,93 @@ el.replaceWith(d);
<summary>
<span>Expand source code</span>
</summary>
<pre><code class="python">def resolve_node_data(self, node_data):
&#34;&#34;&#34;Resolve profile references (@profile) in node data and handle inheritance.&#34;&#34;&#34;
resolved = node_data.copy()
<pre><code class="python">def resolve_node_data(
self,
node_data,
*,
_visited=frozenset(),
_depth=0,
_profiles_snapshot=None,
):
&#34;&#34;&#34;Resolve profile references (@profile) in node data while preserving compose and scalar secrets.&#34;&#34;&#34;
from ..auth_utils import is_compose_expression
# 1. Identify all referenced profiles to support inheritance
referenced_profiles = []
for value in resolved.values():
if isinstance(value, str) and value.startswith(&#34;@&#34;):
referenced_profiles.append(value[1:])
elif isinstance(value, list):
if _depth &gt; MAX_PROFILE_DEPTH:
raise InvalidConfigurationError(&#34;Profile inheritance exceeds maximum depth.&#34;)
profiles = (
_profiles_snapshot
if _profiles_snapshot is not None
else self._get_profiles_snapshot()
)
resolved = deepcopy(node_data)
auth_referenced_profiles = []
for key in (&#34;user&#34;, &#34;password&#34;):
value = resolved.get(key)
if isinstance(value, str):
if not is_compose_expression(value) and value.startswith(&#34;@&#34;):
ref = value[1:]
if key != &#34;password&#34; or ref in profiles:
auth_referenced_profiles.append(ref)
elif isinstance(value, (list, tuple)):
for item in value:
if isinstance(item, str) and item.startswith(&#34;@&#34;):
referenced_profiles.append(item[1:])
if (
isinstance(item, str)
and not is_compose_expression(item)
and item.startswith(&#34;@&#34;)
):
ref = item[1:]
if key != &#34;password&#34; or ref in profiles:
auth_referenced_profiles.append(ref)
def _get_resolved_ref(prof_name):
if prof_name in _visited or _depth &gt;= MAX_PROFILE_DEPTH:
raise InvalidConfigurationError(
f&#34;Circular or overly deep profile reference involving &#39;{prof_name}&#39;.&#34;
)
prof_raw = profiles.get(prof_name)
if not prof_raw:
raise ProfileNotFoundError(f&#34;Profile &#39;{prof_name}&#39; not found.&#34;)
return self.resolve_node_data(
prof_raw,
_visited=_visited | {prof_name},
_depth=_depth + 1,
_profiles_snapshot=profiles,
)
for key, value in list(resolved.items()):
if key == &#34;fallback&#34;:
continue
if key == &#34;password&#34;:
if isinstance(value, str):
if is_compose_expression(value):
continue
if value.startswith(&#34;@&#34;) and value[1:] in profiles:
prof_obj = _get_resolved_ref(value[1:])
resolved[&#34;password&#34;] = prof_obj.get(&#34;password&#34;, &#34;&#34;)
elif isinstance(value, list):
resolved_list = []
for item in value:
if (
isinstance(item, str)
and not is_compose_expression(item)
and item.startswith(&#34;@&#34;)
and item[1:] in profiles
):
with suppress(ProfileNotFoundError):
prof_obj = _get_resolved_ref(item[1:])
if &#34;password&#34; in prof_obj:
resolved_list.append(prof_obj[&#34;password&#34;])
else:
resolved_list.append(item)
resolved[&#34;password&#34;] = resolved_list
continue
# 2. Resolve explicit references
for key, value in resolved.items():
if isinstance(value, str) and value.startswith(&#34;@&#34;):
profile_name = value[1:]
try:
profile = self.get_profile(profile_name, resolve=True)
profile = _get_resolved_ref(profile_name)
resolved[key] = profile.get(key, &#34;&#34;)
except ProfileNotFoundError:
resolved[key] = &#34;&#34;
@@ -326,39 +572,35 @@ el.replaceWith(d);
for item in value:
if isinstance(item, str) and item.startswith(&#34;@&#34;):
profile_name = item[1:]
try:
profile = self.get_profile(profile_name, resolve=True)
with suppress(ProfileNotFoundError):
profile = _get_resolved_ref(profile_name)
if &#34;password&#34; in profile:
resolved_list.append(profile[&#34;password&#34;])
except ProfileNotFoundError:
pass
else:
resolved_list.append(item)
resolved[key] = resolved_list
# 3. Inheritance: Fill empty keys from the first referenced profile
if referenced_profiles:
base_profile_name = referenced_profiles[0]
try:
base_profile = self.get_profile(base_profile_name, resolve=True)
for key, value in base_profile.items():
# Fill if key is missing or empty
if key not in resolved or resolved[key] == &#34;&#34; or resolved[key] == [] or resolved[key] is None:
resolved[key] = value
except ProfileNotFoundError:
pass
dedup_auth_profiles = list(dict.fromkeys(auth_referenced_profiles))
for base_profile_name in dedup_auth_profiles:
with suppress(ProfileNotFoundError):
base_profile = _get_resolved_ref(base_profile_name)
for key in (&#34;user&#34;, &#34;password&#34;, &#34;fallback&#34;):
val = base_profile.get(key)
if (
key not in resolved or resolved[key] in (&#34;&#34;, [], None)
) and val not in (&#34;&#34;, [], None):
resolved[key] = deepcopy(val)
# 4. Handle default protocol
if resolved.get(&#34;protocol&#34;) == &#34;&#34; or resolved.get(&#34;protocol&#34;) is None:
if resolved.get(&#34;protocol&#34;) in (&#34;&#34;, None):
try:
default_profile = self.get_profile(&#34;default&#34;, resolve=True)
resolved[&#34;protocol&#34;] = default_profile.get(&#34;protocol&#34;, &#34;ssh&#34;)
default_profile = _get_resolved_ref(&#34;default&#34;)
resolved[&#34;protocol&#34;] = default_profile.get(&#34;protocol&#34;, &#34;ssh&#34;) or &#34;ssh&#34;
except ProfileNotFoundError:
resolved[&#34;protocol&#34;] = &#34;ssh&#34;
return resolved</code></pre>
</details>
<div class="desc"><p>Resolve profile references (@profile) in node data and handle inheritance.</p></div>
<div class="desc"><p>Resolve profile references (@profile) in node data while preserving compose and scalar secrets.</p></div>
</dd>
<dt id="connpy.services.profile_service.ProfileService.update_profile"><code class="name flex">
<span>def <span class="ident">update_profile</span></span>(<span>self, name, data)</span>
@@ -370,18 +612,28 @@ el.replaceWith(d);
</summary>
<pre><code class="python">def update_profile(self, name, data):
&#34;&#34;&#34;Update an existing profile.&#34;&#34;&#34;
if _supports_transactional_update(self.config):
def _mut(cfg):
if name not in cfg.profiles:
raise ProfileNotFoundError(f&#34;Profile &#39;{name}&#39; not found.&#34;)
updated_data = deepcopy(cfg.profiles[name])
updated_data.update(deepcopy(data))
filtered_data = {
k: v for k, v in updated_data.items() if k in self._ALLOWED_KEYS
}
self._validate_profile_payload(filtered_data)
cfg._profiles_add_unlocked(id=name, **filtered_data)
self.config.transactional_update(_mut)
return
if name not in self.config.profiles:
raise ProfileNotFoundError(f&#34;Profile &#39;{name}&#39; not found.&#34;)
# Merge with existing data
existing = self.get_profile(name, resolve=False)
updated_data = existing.copy()
updated_data = deepcopy(existing)
updated_data.update(data)
# Filter data to match _profiles_add signature
allowed_keys = {&#34;host&#34;, &#34;options&#34;, &#34;logs&#34;, &#34;password&#34;, &#34;port&#34;, &#34;protocol&#34;, &#34;user&#34;, &#34;tags&#34;, &#34;jumphost&#34;}
filtered_data = {k: v for k, v in updated_data.items() if k in allowed_keys}
filtered_data = {k: v for k, v in updated_data.items() if k in self._ALLOWED_KEYS}
self._validate_profile_payload(filtered_data)
self.config._profiles_add(id=name, **filtered_data)
self.config._saveconfig(self.config.file)</code></pre>
</details>
+100 -20
View File
@@ -65,7 +65,7 @@ el.replaceWith(d);
</dd>
<dt id="connpy.services.provider.ServiceProvider"><code class="flex name class">
<span>class <span class="ident">ServiceProvider</span></span>
<span>(</span><span>config, mode='local', remote_host=None)</span>
<span>(</span><span>config, mode='local', remote_host=None, enforce_context: bool = True)</span>
</code></dt>
<dd>
<details class="source">
@@ -75,16 +75,22 @@ el.replaceWith(d);
<pre><code class="python">class ServiceProvider:
&#34;&#34;&#34;Dynamic service backend. Transparently provides local or remote services.&#34;&#34;&#34;
def __init__(self, config, mode=&#34;local&#34;, remote_host=None):
def __init__(self, config, mode=&#34;local&#34;, remote_host=None, enforce_context: bool = True):
from ..auth_utils import AuthAttemptTracker
self.mode = mode
self.config = config
self.remote_host = remote_host
self.enforce_context = bool(enforce_context)
self.auth_tracker = AuthAttemptTracker()
self.nodes = None
self._system = None
self._execution = None
self._import_export = None
self._sync = None
self._users = None
self._ai = None
self.set_enforce_context(self.enforce_context)
if mode == &#34;local&#34;:
self._init_local()
@@ -93,14 +99,26 @@ el.replaceWith(d);
else:
raise ValueError(f&#34;Unknown service mode: {mode}&#34;)
def _init_local(self):
from .node_service import NodeService
from .profile_service import ProfileService
from .config_service import ConfigService
from .plugin_service import PluginService
from .context_service import ContextService
def set_enforce_context(self, enforce_context: bool) -&gt; None:
self.enforce_context = bool(enforce_context)
if self.config is not None and not hasattr(self.config, &#34;assert_called&#34;):
self.config._enforce_context = self.enforce_context
for svc in (
getattr(self, &#34;nodes&#34;, None),
self._execution,
self._import_export,
):
if svc is not None and hasattr(svc, &#34;enforce_context&#34;):
svc.enforce_context = self.enforce_context
self.nodes = NodeService(self.config)
def _init_local(self):
from .config_service import ConfigService
from .context_service import ContextService
from .node_service import NodeService
from .plugin_service import PluginService
from .profile_service import ProfileService
self.nodes = NodeService(self.config, enforce_context=self.enforce_context)
self.profiles = ProfileService(self.config)
self.config_svc = ConfigService(self.config)
self.plugins = PluginService(self.config)
@@ -110,18 +128,28 @@ el.replaceWith(d);
# Allow ConfigService to work locally so the user can revert the mode
from .config_service import ConfigService
from .context_service import ContextService
self.config_svc = ConfigService(self.config)
self.context = ContextService(self.config)
if not self.remote_host:
raise InvalidConfigurationError(&#34;Remote host must be specified in remote mode&#34;)
import grpc
import os
import grpc
from ..grpc_layer.stubs import (
NodeStub, ProfileStub, PluginStub, AIStub,
ExecutionStub, ImportExportStub, SystemStub,
ConfigStub, AuthClientInterceptor, AuthStub
AIStub,
AuthClientInterceptor,
AuthStub,
ConfigStub,
ExecutionStub,
ImportExportStub,
NodeStub,
PluginStub,
ProfileStub,
SystemStub,
)
def get_token():
@@ -130,11 +158,8 @@ el.replaceWith(d);
return env_token
token_path = os.path.join(self.config.defaultdir, &#34;.token&#34;)
if os.path.exists(token_path):
try:
with open(token_path, &#34;r&#34;) as f:
with suppress(OSError, UnicodeDecodeError), open(token_path) as f:
return f.read().strip()
except Exception:
pass
return None
channel = grpc.insecure_channel(self.remote_host)
@@ -151,7 +176,12 @@ el.replaceWith(d);
self.plugins = PluginStub(channel, remote_host=self.remote_host)
self.ai = AIStub(channel, remote_host=self.remote_host)
self.system = SystemStub(channel, remote_host=self.remote_host)
self.execution = ExecutionStub(channel, remote_host=self.remote_host)
self.execution = ExecutionStub(
channel,
remote_host=self.remote_host,
config=self.config,
node_stub=self.nodes,
)
self.import_export = ImportExportStub(channel, remote_host=self.remote_host)
self.auth = AuthStub(channel, remote_host=self.remote_host)
@@ -170,7 +200,13 @@ el.replaceWith(d);
def execution(self):
if self._execution is None and self.mode == &#34;local&#34;:
from .execution_service import ExecutionService
self._execution = ExecutionService(self.config)
params = inspect.signature(ExecutionService.__init__).parameters
kwargs = {}
if &#34;attempt_tracker&#34; in params:
kwargs[&#34;attempt_tracker&#34;] = self.auth_tracker
if &#34;enforce_context&#34; in params:
kwargs[&#34;enforce_context&#34;] = self.enforce_context
self._execution = ExecutionService(self.config, **kwargs)
return self._execution
@execution.setter
@@ -181,6 +217,12 @@ el.replaceWith(d);
def import_export(self):
if self._import_export is None and self.mode == &#34;local&#34;:
from .import_export_service import ImportExportService
params = inspect.signature(ImportExportService.__init__).parameters
if &#34;enforce_context&#34; in params:
self._import_export = ImportExportService(
self.config, enforce_context=self.enforce_context
)
else:
self._import_export = ImportExportService(self.config)
return self._import_export
@@ -249,7 +291,13 @@ def ai(self):
def execution(self):
if self._execution is None and self.mode == &#34;local&#34;:
from .execution_service import ExecutionService
self._execution = ExecutionService(self.config)
params = inspect.signature(ExecutionService.__init__).parameters
kwargs = {}
if &#34;attempt_tracker&#34; in params:
kwargs[&#34;attempt_tracker&#34;] = self.auth_tracker
if &#34;enforce_context&#34; in params:
kwargs[&#34;enforce_context&#34;] = self.enforce_context
self._execution = ExecutionService(self.config, **kwargs)
return self._execution</code></pre>
</details>
<div class="desc"></div>
@@ -264,6 +312,12 @@ def execution(self):
def import_export(self):
if self._import_export is None and self.mode == &#34;local&#34;:
from .import_export_service import ImportExportService
params = inspect.signature(ImportExportService.__init__).parameters
if &#34;enforce_context&#34; in params:
self._import_export = ImportExportService(
self.config, enforce_context=self.enforce_context
)
else:
self._import_export = ImportExportService(self.config)
return self._import_export</code></pre>
</details>
@@ -315,6 +369,31 @@ def users(self):
<div class="desc"></div>
</dd>
</dl>
<h3>Methods</h3>
<dl>
<dt id="connpy.services.provider.ServiceProvider.set_enforce_context"><code class="name flex">
<span>def <span class="ident">set_enforce_context</span></span>(<span>self, enforce_context: bool) ‑> None</span>
</code></dt>
<dd>
<details class="source">
<summary>
<span>Expand source code</span>
</summary>
<pre><code class="python">def set_enforce_context(self, enforce_context: bool) -&gt; None:
self.enforce_context = bool(enforce_context)
if self.config is not None and not hasattr(self.config, &#34;assert_called&#34;):
self.config._enforce_context = self.enforce_context
for svc in (
getattr(self, &#34;nodes&#34;, None),
self._execution,
self._import_export,
):
if svc is not None and hasattr(svc, &#34;enforce_context&#34;):
svc.enforce_context = self.enforce_context</code></pre>
</details>
<div class="desc"></div>
</dd>
</dl>
</dd>
</dl>
</section>
@@ -340,6 +419,7 @@ def users(self):
<li><code><a title="connpy.services.provider.ServiceProvider.ai" href="#connpy.services.provider.ServiceProvider.ai">ai</a></code></li>
<li><code><a title="connpy.services.provider.ServiceProvider.execution" href="#connpy.services.provider.ServiceProvider.execution">execution</a></code></li>
<li><code><a title="connpy.services.provider.ServiceProvider.import_export" href="#connpy.services.provider.ServiceProvider.import_export">import_export</a></code></li>
<li><code><a title="connpy.services.provider.ServiceProvider.set_enforce_context" href="#connpy.services.provider.ServiceProvider.set_enforce_context">set_enforce_context</a></code></li>
<li><code><a title="connpy.services.provider.ServiceProvider.sync" href="#connpy.services.provider.ServiceProvider.sync">sync</a></code></li>
<li><code><a title="connpy.services.provider.ServiceProvider.system" href="#connpy.services.provider.ServiceProvider.system">system</a></code></li>
<li><code><a title="connpy.services.provider.ServiceProvider.users" href="#connpy.services.provider.ServiceProvider.users">users</a></code></li>
+2 -1
View File
@@ -435,8 +435,9 @@ el.replaceWith(d);
</details>
<div class="desc"><p>Business logic for Google Drive synchronization.</p>
<p>Initialize the service.</p>
<h2 id="args">Args</h2>
<dl>
<dt>Args</dt>
<dt>-----=</dt>
<dt><strong><code>config</code></strong></dt>
<dd>An instance of configfile (or None to instantiate a new one/use global context).</dd>
</dl></div>
+2 -1
View File
@@ -142,8 +142,9 @@ el.replaceWith(d);
</details>
<div class="desc"><p>Business logic for application lifecycle (API, processes).</p>
<p>Initialize the service.</p>
<h2 id="args">Args</h2>
<dl>
<dt>Args</dt>
<dt>-----=</dt>
<dt><strong><code>config</code></strong></dt>
<dd>An instance of configfile (or None to instantiate a new one/use global context).</dd>
</dl></div>