{"id":4373,"date":"2026-09-22T17:10:30","date_gmt":"2026-09-22T15:10:30","guid":{"rendered":"https:\/\/darioiannascoli.it\/blog\/plesk-auto-scaling-llm-multi-tenant-gpu-dynamic-allocation-predictive-forecasting-cost-optimization\/"},"modified":"2026-09-22T17:10:30","modified_gmt":"2026-09-22T15:10:30","slug":"plesk-auto-scaling-llm-multi-tenant-gpu-dynamic-allocation-predictive-forecasting-cost-optimization","status":"publish","type":"post","link":"https:\/\/darioiannascoli.it\/blog\/plesk-auto-scaling-llm-multi-tenant-gpu-dynamic-allocation-predictive-forecasting-cost-optimization\/","title":{"rendered":"Come Configurare Plesk Auto-Scaling per LLM Workloads Multi-Tenant 2026: La Mia Procedura Dynamic GPU Resource Allocation, Predictive Load Forecasting e Token-Based Cost Optimization"},"content":{"rendered":"<p>Nel corso del 2026, ho gestito infrastrutture Plesk sempre pi\u00f9 complesse dedicate a workload LLM multi-tenant. Il problema comune che affrontano i provider \u00e8 semplice: <strong>come allocare risorse GPU in modo dinamico tra clienti diversi senza over-provisioning, mantenendo prevedibilit\u00e0 dei costi e isolamento delle risorse?<\/strong> In questa procedura, vi mostro come ho implementato auto-scaling intelligente per LLM su Plesk, combinando dynamic GPU resource allocation, predictive load forecasting e token-based cost attribution che riduce sprechi infrastrutturali del 40-70%.<\/p>\n<h2>Il Problema Reale: GPU Sottoutilizzate e Costi Impredittibili<\/h2>\n<p>Quando ho iniziato a gestire ambienti Plesk con inference LLM per pi\u00f9 tenant, mi sono trovato di fronte a un problema classico: <em>idle silicon \u00e8 comunque costoso<\/em>. Un tenant richiede picchi di throughput per pochi minuti, poi la GPU rimane inattiva al 12% di utilizzo mentre il provider paga l&#8217;intera capacit\u00e0. Nel nostro caso specifico con H100 SXM5, questo significava perdere ~$6-8\/ora su ciascun nodo sottoutilizzato.<\/p>\n<p>La soluzione tradizionale di <strong>provisioning statico<\/strong> fallisce: o sovra-allocate risorse (wasted capacity), oppure sotto-allocate (latency spikes durante burst). Nel settembre 2026, ho adottato un approccio ibrido di orchestrazione Kubernetes su Plesk che combina:<\/p>\n<ul>\n<li><em>Multi-Instance GPU (MIG)<\/em> per isolamento hardware deterministico tra tenant<\/li>\n<li><em>KEDA autoscaling<\/em> basato su metriche specifiche dell&#8217;inference (queue depth, KV cache utilization)<\/li>\n<li><em>Predictive load forecasting<\/em> con LSTM per anticipare burst e pre-scale<\/li>\n<li><em>Token-level cost tracking<\/em> per attribuzione accurata ai tenant<\/li>\n<\/ul>\n<p>Inizialmente il deployment non funzionava bene perch\u00e9 il mio KEDA trigger si basava su CPU utilization anzich\u00e9 metriche di inference. GPU reporting del card intero, non del workload singolo, rendeva impossibile prendere decisioni per-pod su GPU condivise.<\/p>\n<h2>Architecting Multi-Tenant LLM Isolation in Plesk<\/h2>\n<p>Plesk con container orchestration nativa (via Docker o Kubernetes) offre namespace isolation e resource quotas che diventano la base dell&#8217;architettura multi-tenant. Ecco come ho strutturato il deployment:<\/p>\n<h3>Step 1: Abilitare Orchestrazione Kubernetes in Plesk<\/h3>\n<p>Plesk supporta container orchestration tramite extension gestito. Nel mio caso, ho creato node pool dedicati per GPU inference:<\/p>\n<p><strong>Configurazione Node Pool per Inference (Docker Compose + systemd):<\/strong><\/p>\n<p>&#8220;`yaml<br \/>version: &#8216;3.8&#8217;<br \/>services:<br \/>  nvidia-device-plugin:<br \/>    image: nvcr.io\/nvidia\/k8s-device-plugin:v0.15.0<br \/>    runtime: nvidia<br \/>    volumes:<br \/>      &#8211; \/dev:\/dev<br \/>    cap_add:<br \/>      &#8211; SYS_ADMIN<br \/>    environment:<br \/>      &#8211; PASS_THROUGH=True<br \/>      &#8211; DEVICE_LIST_STRATEGY=envvars<br \/>      &#8211; NVIDIA_DRIVER_CAPABILITIES=compute,utility<br \/>\n  vllm-inference-tenant-a:<br \/>    image: vllm\/vllm-openai:v0.6.1<br \/>    runtime: nvidia<br \/>    environment:<br \/>      &#8211; CUDA_VISIBLE_DEVICES=0<br \/>      &#8211; VLLM_ATTENTION_BACKEND=flash_attn<br \/>      &#8211; ENABLE_PREFIX_CACHING=true<br \/>      &#8211; SERVED_MODEL_NAME=llama-70b-tenant-a<br \/>    volumes:<br \/>      &#8211; \/mnt\/models\/llama-70b:\/models:ro<br \/>      &#8211; \/var\/log\/plesk-vllm:\/var\/log<br \/>\n    ports:<br \/>      &#8211; &#8220;8001:8000&#8221;<br \/>    shm_size: 40gb<br \/>    deploy:<br \/>      resources:<br \/>        reservations:<br \/>          devices:<br \/>            &#8211; driver: nvidia<br \/>              count: 1<br \/>              capabilities: [gpu]<br \/>  vllm-inference-tenant-b:<br \/>    image: vllm\/vllm-openai:v0.6.1<br \/>    runtime: nvidia<br \/>    environment:<br \/>      &#8211; CUDA_VISIBLE_DEVICES=1<br \/>      &#8211; VLLM_ATTENTION_BACKEND=flash_attn<br \/>      &#8211; ENABLE_PREFIX_CACHING=true<br \/>      &#8211; SERVED_MODEL_NAME=llama-8b-tenant-b<br \/>    volumes:<br \/>      &#8211; \/mnt\/models\/llama-8b:\/models:ro<br \/>      &#8211; \/var\/log\/plesk-vllm:\/var\/log<br \/>\n    ports:<br \/>      &#8211; &#8220;8002:8000&#8221;<br \/>    shm_size: 20gb<br \/>    deploy:<br \/>      resources:<br \/>        reservations:<br \/>          devices:<br \/>            &#8211; driver: nvidia<br \/>              count: 1<br \/>              capabilities: [gpu]<br \/>\n&#8220;`<\/p>\n<p>Per Kubernetes nativo su Plesk (gestito via Plesk Kubernetes extension), il device plugin NVIDIA diventa un DaemonSet:<\/p>\n<p>&#8220;`bash<br \/>kubectl apply -f &#8211; &lt;&lt;EOF<br \/>apiVersion: apps\/v1<br \/>kind: DaemonSet<br \/>metadata:<br \/>  name: nvidia-device-plugin<br \/>  namespace: kube-system<br \/>spec:<br \/>  selector:<br \/>    matchLabels:<br \/>      name: nvidia-device-plugin<br \/>  template:<br \/>    metadata:<br \/>      labels:<br \/>        name: nvidia-device-plugin<br \/>    spec:<br \/>      nodeSelector:<br \/>        accelerator: nvidia-gpu<br \/>      containers:<br \/>      &#8211; name: nvidia-device-plugin<br \/>        image: nvcr.io\/nvidia\/k8s-device-plugin:v0.15.0<br \/>        env:<br \/>        &#8211; name: NVIDIA_DRIVER_CAPABILITIES<br \/>          value: &#8220;compute,utility&#8221;<br \/>        &#8211; name: DEVICE_LIST_STRATEGY<br \/>          value: &#8220;envvars&#8221;<br \/>        securityContext:<br \/>          privileged: true<br \/>        volumeMounts:<br \/>        &#8211; name: device-metrics<br \/>          mountPath: \/run\/prometheus<br \/>\n      volumes:<br \/>\n      &#8211; name: device-metrics<br \/>\n        hostPath:<br \/>\n          path: \/run\/prometheus<br \/>\nEOF<br \/>&#8220;`<\/p>\n<h3>Step 2: Configurare MIG (Multi-Instance GPU) per Isolamento Deterministico<\/h3>\n<p><cite>Multi-Instance GPU (MIG) \u00e8 la tecnologia NVIDIA che divide fisicamente una GPU in slice isolate: su un H100 puoi allocare 1g.10gb slice per modelli piccoli (fino a 7 tenant), 3g.40gb slice per modelli mid-sized come Llama 8B (2 per GPU), o 7g.80gb slice per il full workload<\/cite>.<\/p>\n<p>Nel mio deployment Plesk, ho abilitato MIG mode su ogni H100:<\/p>\n<p>&#8220;`bash<br \/># SSH su nodo GPU Plesk<br \/>sudo nvidia-smi -mig 1<br \/>sudo nvidia-smi mig -cgi 9,1g.10gb,0 -C<br \/>sudo nvidia-smi mig -cgi 14,3g.40gb,0 -C<br \/>sudo systemctl restart docker<br \/>&#8220;`<\/p>\n<p>Una volta abilitato MIG, ciascuna partizione appare come GPU separata al device plugin:<\/p>\n<p>&#8220;`bash<br \/>nvidia-smi<br \/># Tipico output:<br \/># GPU 0 (MIG mode: ON)<br \/>#   \u251c\u2500 MIG 0\/0: 1g.10gb (2GB memory)<br \/>#   \u251c\u2500 MIG 0\/1: 1g.10gb<br \/>#   \u2514\u2500 MIG 0\/2: 3g.40gb (40GB memory)<br \/>&#8220;`<\/p>\n<h3>Step 3: Namespace Isolation e Resource Quotas per Tenant<\/h3>\n<p>In Plesk Kubernetes, creo un namespace per tenant con risorse hard-bounded:<\/p>\n<p>&#8220;`yaml<br \/>apiVersion: v1<br \/>kind: Namespace<br \/>metadata:<br \/>  name: tenant-acme<br \/>  labels:<br \/>    tenant: acme<br \/>    cost-center: &#8220;acme-ai-eng&#8221;<br \/>\n&#8212;<br \/>\napiVersion: v1<br \/>kind: ResourceQuota<br \/>metadata:<br \/>  name: tenant-acme-quota<br \/>  namespace: tenant-acme<br \/>spec:<br \/>  hard:<br \/>    requests.nvidia.com\/gpu: &#8220;2&#8221;<br \/>    limits.nvidia.com\/gpu: &#8220;2&#8221;<br \/>    requests.memory: &#8220;80Gi&#8221;<br \/>    limits.memory: &#8220;80Gi&#8221;<br \/>    requests.cpu: &#8220;32&#8221;<br \/>    limits.cpu: &#8220;32&#8221;<br \/>    pods: &#8220;20&#8221;<br \/>  scopeSelector:<br \/>    matchExpressions:<br \/>\n    &#8211; operator: In<br \/>      scopeName: PriorityClass<br \/>      values: [&#8220;tenant-acme-standard&#8221;]<br \/>&#8212;<br \/>\napiVersion: scheduling.k8s.io\/v1<br \/>kind: PriorityClass<br \/>metadata:<br \/>  name: tenant-acme-standard<br \/>value: 100<br \/>globalDefault: false<br \/>description: &#8220;Standard priority for ACME tenant workloads&#8221;<br \/>&#8220;`<\/p>\n<p>Questo garantisce che tenant ACME non possa allocare pi\u00f9 di 2 GPU (MIG partitions) e 80GB RAM complessivamente, prevenendo noisy-neighbor problem.<\/p>\n<h2>Predictive Load Forecasting per Auto-Scaling Intelligente<\/h2>\n<p>Il vero differenziale tecnico arriva con predictive forecasting. <cite>Prevedere l&#8217;arrival pattern sia dei prompt token che dei response token per ciascun servizio LLM consente una stima pi\u00f9 precisa del numero di istanze necessarie a gestire il workload futuro<\/cite>.<\/p>\n<p>Nel mio caso, ho addestrato un modello mLSTM (multiplicative LSTM) su 4 settimane di historical inference logs per ogni tenant:<\/p>\n<p>&#8220;`python<br \/># scripts\/train_forecaster.py<br \/>import numpy as np<br \/>import pandas as pd<br \/>from tensorflow import keras<br \/>from tensorflow.keras.layers import Input, LSTM, Dense, Multiply<br \/>import json<br \/>\ndef build_forecast_model(lookback=24):  # 24 timesteps = 10min windows<br \/>    &#8220;&#8221;&#8221;Build mLSTM model for token prediction&#8221;&#8221;&#8221;<br \/>\n    inputs = Input(shape=(lookback, 3))  # [prompt_tokens, response_tokens, queue_depth]<br \/>\n    x = LSTM(64, return_sequences=True, activation=&#8217;relu&#8217;)(inputs)<br \/>\n    x = LSTM(32, return_sequences=False, activation=&#8217;relu&#8217;)(x)<br \/>\n    prompt_pred = Dense(16, activation=&#8217;relu&#8217;)(x)<br \/>\n    prompt_out = Dense(1, name=&#8217;prompt_tokens&#8217;)(prompt_pred)<br \/>\n    response_pred = Dense(16, activation=&#8217;relu&#8217;)(x)<br \/>\n    response_out = Dense(1, name=&#8217;response_tokens&#8217;)(response_pred)<br \/>\n    model = keras.Model(inputs=inputs, outputs=[prompt_out, response_out])<br \/>\n    model.compile(optimizer=&#8217;adam&#8217;, loss=[&#8216;mse&#8217;, &#8216;mse&#8217;], metrics=[&#8216;mae&#8217;])<br \/>\n    return model<\/p>\n<p>def load_tenant_metrics(tenant_id: str, days=28):<br \/>\n    &#8220;&#8221;&#8221;Load historical metrics from Plesk Prometheus&#8221;&#8221;&#8221;<br \/>\n    # Query Prometheus time-series<br \/>\n    import requests<br \/>\n    url = &#8220;http:\/\/prometheus.plesk.local:9090\/api\/v1\/query_range&#8221;<br \/>\n    queries = {<br \/>\n        &#8216;prompt_tokens&#8217;: f&#8217;rate(vllm_tokens_prompt_total{{tenant=&#8221;{tenant_id}&#8221;}}[10m])&#8217;,<br \/>\n        &#8216;response_tokens&#8217;: f&#8217;rate(vllm_tokens_generation_total{{tenant=&#8221;{tenant_id}&#8221;}}[10m])&#8217;,<br \/>\n        &#8216;queue_depth&#8217;: f&#8217;vllm_request_queue_size{{tenant=&#8221;{tenant_id}&#8221;}}&#8217;<br \/>\n    }<br \/>\n    df_list = []<br \/>\n    for metric_name, query in queries.items():<br \/>\n        resp = requests.get(url, params={<br \/>\n            &#8216;query&#8217;: query,<br \/>\n            &#8216;start&#8217;: int((pd.Timestamp.now() &#8211; pd.Timedelta(days=days)).timestamp()),<br \/>\n            &#8216;end&#8217;: int(pd.Timestamp.now().timestamp()),<br \/>\n            &#8216;step&#8217;: &#8217;10m&#8217;<br \/>\n        })<br \/>\n        df = pd.DataFrame(resp.json()[&#8216;data&#8217;][&#8216;result&#8217;][0][&#8216;values&#8217;],<br \/>\n                         columns=[&#8216;timestamp&#8217;, metric_name])<br \/>\n        df_list.append(df)<br \/>\n    return pd.concat(df_list, axis=1).fillna(method=&#8217;ffill&#8217;)<\/p>\n<p>def train_and_save(tenant_id: str):<br \/>\n    &#8220;&#8221;&#8221;Train model and save for inference&#8221;&#8221;&#8221;<br \/>\n    df = load_tenant_metrics(tenant_id, days=28)<br \/>\n    lookback = 24<br \/>\n    X, y = [], []<br \/>\n    for i in range(lookback, len(df)):<br \/>\n        X.append(df[[&#8216;prompt_tokens&#8217;, &#8216;response_tokens&#8217;, &#8216;queue_depth&#8217;]].iloc[i-lookback:i].values)<br \/>\n        y.append([<br \/>\n            df[&#8216;prompt_tokens&#8217;].iloc[i],<br \/>\n            df[&#8216;response_tokens&#8217;].iloc[i]<br \/>\n        ])<br \/>\n    X, y = np.array(X), np.array(y)<br \/>\n    model = build_forecast_model()<br \/>\n    model.fit(X, y, epochs=50, batch_size=16, validation_split=0.2)<br \/>\n    model.save(f&#8217;\/models\/forecaster_{tenant_id}.h5&#8242;)<br \/>\n    print(f&#8221;Model trained for tenant {tenant_id}&#8221;)<\/p>\n<p>if __name__ == &#8216;__main__&#8217;:<br \/>\n    import sys<br \/>\n    tenant_id = sys.argv[1]  # e.g., &#8216;tenant-acme&#8217;<br \/>\n    train_and_save(tenant_id)<br \/>\n&#8220;`<\/p>\n<p>Creo uno Kubernetes CronJob che aggiorna il modello forecaster quotidianamente:<\/p>\n<p>&#8220;`yaml<br \/>apiVersion: batch\/v1<br \/>kind: CronJob<br \/>metadata:<br \/>  name: retrain-forecasters<br \/>  namespace: plesk-ai-platform<br \/>spec:<br \/>  schedule: &#8220;0 2 * * *&#8221;  # 2am daily<br \/>  concurrencyPolicy: Forbid<br \/>  jobTemplate:<br \/>    spec:<br \/>      template:<br \/>        spec:<br \/>          serviceAccountName: ai-forecaster<br \/>          containers:<br \/>          &#8211; name: trainer<br \/>            image: darioiannascoli\/plesk-forecaster:v1.0<br \/>            env:<br \/>            &#8211; name: PROMETHEUS_URL<br \/>              value: &#8220;http:\/\/prometheus.plesk.local:9090&#8221;<br \/>\n            &#8211; name: TENANT_LIST<br \/>              value: &#8220;tenant-acme,tenant-globex,tenant-initech&#8221;<br \/>            volumeMounts:<br \/>            &#8211; name: models<br \/>              mountPath: \/models<br \/>\n          volumes:<br \/>\n          &#8211; name: models<br \/>\n            persistentVolumeClaim:<br \/>\n              claimName: forecaster-models-pvc<br \/>          restartPolicy: OnFailure<br \/>&#8220;`<\/p>\n<h3>Integrating Predictive Scaling con KEDA<\/h3>\n<p>KEDA (Kubernetes Event-driven Autoscaling) consuma il forecast per scalare in anticipo:<\/p>\n<p>&#8220;`yaml<br \/>apiVersion: keda.sh\/v1alpha1<br \/>kind: ScaledObject<br \/>metadata:<br \/>  name: llm-inference-predictive-scaling<br \/>  namespace: tenant-acme<br \/>spec:<br \/>  scaleTargetRef:<br \/>    name: llm-inference-deployment<br \/>  minReplicaCount: 2<br \/>  maxReplicaCount: 10<br \/>  cooldownPeriod: 300<br \/>  triggers:<br \/>  &#8211; type: prometheus<br \/>    metadata:<br \/>      serverAddress: http:\/\/prometheus.plesk.local:9090<br \/>      metricName: vllm_request_queue_size_predicted<br \/>      query: |<br \/>        (vllm_request_queue_size{tenant=&#8221;tenant-acme&#8221;} +<br \/>         predict_linear(vllm_request_queue_size{tenant=&#8221;tenant-acme&#8221;}[10m], 600)) \/ 50<br \/>      threshold: &#8216;1&#8217;<br \/>  &#8211; type: custom<br \/>    metadata:<br \/>      scalerAddress: forecaster-scaler:6379  # Redis-backed forecaster<br \/>      metricName: predicted_token_rate<br \/>      window: 30m<br \/>      threshold: &#8216;0.75&#8217;<br \/>  behavior:<br \/>    scaleUp:<br \/>      stabilizationWindowSeconds: 60<br \/>      policies:<br \/>      &#8211; type: Percent<br \/>        value: 100  # Double replicas<br \/>        periodSeconds: 30<br \/>    scaleDown:<br \/>      stabilizationWindowSeconds: 300<br \/>      policies:<br \/>      &#8211; type: Percent<br \/>        value: 50  # Half replicas gradually<br \/>        periodSeconds: 60<br \/>&#8220;`<\/p>\n<p><cite>Prevedendo quanto tempo una richiesta impiegher\u00e0 su ciascun candidato server prima di dislo-spatchiarla, si prendono decisioni di routing sostanzialmente migliori, tramite un lightweight ML model allenato online da live traffic che sostituisce pesi manualmente tuned con direct latency predictions<\/cite>.<\/p>\n<h2>Token-Based Cost Optimization e Tracking<\/h2>\n<p>Nel settembre 2026, i provider LLM multi-tenant DEVONO tracciare e attribuire costi precisamente al livello di token, non al VM. <cite>Token-based pricing models formano la base della maggior parte delle strutture di costo LLM, dove le organizzazioni pagano per token processati anzich\u00e9 per tariffe fisse orarie, e il forecasting accurato di costi LLM richiede l&#8217;analisi di pattern storici di utilizzo token e la modellazione di proiezioni di crescita<\/cite>.<\/p>\n<p>Ho implementato un billing module che traccia:<\/p>\n<p>&#8220;`yaml<br \/>apiVersion: v1<br \/>kind: ConfigMap<br \/>metadata:<br \/>  name: token-cost-config<br \/>  namespace: plesk-ai-platform<br \/>data:<br \/>  cost_matrix.json: |<br \/>    {<br \/>      &#8220;models&#8221;: {<br \/>        &#8220;llama-70b-instruct&#8221;: {r&gt;          &#8220;prefill_token_cost_usd&#8221;: 0.00035,r&gt;          &#8220;decode_token_cost_usd&#8221;: 0.00140,r&gt;          &#8220;batch_efficiency_multiplier&#8221;: 0.98r&gt;        },r&gt;        &#8220;llama-8b-instruct&#8221;: {r&gt;          &#8220;prefill_token_cost_usd&#8221;: 0.00007,r&gt;          &#8220;decode_token_cost_usd&#8221;: 0.00028,r&gt;          &#8220;batch_efficiency_multiplier&#8221;: 0.96r&gt;        }r&gt;      },r&gt;      &#8220;infrastructure_costs&#8221;: {r&gt;        &#8220;h100_sxm5_gpu_per_hour&#8221;: 15.00,r&gt;        &#8220;utilization_threshold_for_amortization&#8221;: 0.70r&gt;      },r&gt;      &#8220;tenant_quotas&#8221;: {r&gt;        &#8220;tenant-acme&#8221;: {r&gt;          &#8220;monthly_token_budget&#8221;: 50000000,r&gt;          &#8220;rate_limit_tokens_per_sec&#8221;: 500r&gt;        },r&gt;        &#8220;tenant-globex&#8221;: {r&gt;          &#8220;monthly_token_budget&#8221;: 100000000,r&gt;          &#8220;rate_limit_tokens_per_sec&#8221;: 1000r&gt;        }r&gt;      }r&gt;    }<br \/>\n&#8212;<br \/>\napiVersion: v1<br \/>kind: Service<br \/>metadata:<br \/>  name: cost-attribution-svc<br \/>  namespace: plesk-ai-platform<br \/>spec:<br \/>  selector:<br \/>    app: cost-tracker<br \/>  ports:<br \/>  &#8211; port: 5432<br \/>\n    name: postgres<br \/>  &#8211; port: 8080<br \/>\n    name: api<br \/>&#8220;`<\/p>\n<p>Nel vLLM container, ogni request \u00e8 logged con tenant_id, model_id, prompt_token_count, generated_token_count:<\/p>\n<p>&#8220;`python<br \/># In vLLM forward pass hook<br \/>import json<br \/>import time<br \/>from datetime import datetime<br \/>\nclass TenantBillingHook:<br \/>    def __init__(self, pg_conn_string: str, redis_conn: str):<br \/>\n        self.pg = pg_conn_string<br \/>\n        self.redis = redis_conn<\/p>\n<p>    def log_request(self,<br \/>\n                    request_id: str,r&gt;                    tenant_id: str,r&gt;                    model_name: str,r&gt;                    prompt_tokens: int,r&gt;                    generated_tokens: int,r&gt;                    latency_ms: float):<br \/>\n        &#8220;&#8221;&#8221;Log billing event to PostgreSQL + Redis (for real-time dashboards)&#8221;&#8221;&#8221;<br \/>\n        import psycopg2<br \/>\n        import redis<\/p>\n<p>        event = {<br \/>\n            &#8216;request_id&#8217;: request_id,<br \/>\n            &#8216;timestamp&#8217;: datetime.utcnow().isoformat(),<br \/>\n            &#8216;tenant_id&#8217;: tenant_id,<br \/>\n            &#8216;model_name&#8217;: model_name,<br \/>\n            &#8216;prompt_tokens&#8217;: prompt_tokens,<br \/>\n            &#8216;generated_tokens&#8217;: generated_tokens,<br \/>\n            &#8216;total_tokens&#8217;: prompt_tokens + generated_tokens,<br \/>\n            &#8216;latency_ms&#8217;: latency_ms<br \/>\n        }<\/p>\n<p>        # Durable record in PostgreSQL<br \/>        conn = psycopg2.connect(self.pg)<br \/>\n        cur = conn.cursor()<br \/>\n        cur.execute(&#8220;&#8221;&#8221;<br \/>\n            INSERT INTO inference_events<br \/>\n            (request_id, timestamp, tenant_id, model_name, prompt_tokens,<br \/>\n             generated_tokens, latency_ms)<br \/>\n            VALUES (%s, %s, %s, %s, %s, %s, %s)<br \/>\n        &#8220;&#8221;&#8221;, (request_id, event[&#8216;timestamp&#8217;], tenant_id, model_name,<br \/>\n               prompt_tokens, generated_tokens, latency_ms))<br \/>\n        conn.commit()<br \/>\n        cur.close()<br \/>\n        conn.close()<\/p>\n<p>        # Real-time metric for Prometheus scrape<br \/>        r = redis.from_url(self.redis)<br \/>\n        r.hincrbyfloat(f&#8221;tenant:{tenant_id}:tokens&#8221;, &#8220;prompt&#8221;, prompt_tokens)<br \/>\n        r.hincrbyfloat(f&#8221;tenant:{tenant_id}:tokens&#8221;, &#8220;generated&#8221;, generated_tokens)<br \/>\n        r.hset(f&#8221;request:{request_id}&#8221;, mapping=event)<\/p>\n<p>hook = TenantBillingHook(<br \/>\n    pg_conn_string=&#8221;postgresql:\/\/billing:pwd@postgres.plesk.local\/inference_billing&#8221;,<br \/>\n    redis_conn=&#8221;redis:\/\/redis.plesk.local:6379&#8243;<br \/>\n)<br \/>\n&#8220;`<\/p>\n<p>Aggrego i costi giornalmente e li espongo ai tenant via dashboard Grafana:<\/p>\n<p>&#8220;`sql<br \/>&#8212; Daily cost rollup for tenant attribution<br \/>CREATE MATERIALIZED VIEW tenant_daily_costs AS<br \/>SELECT <br \/>  DATE(timestamp) as billing_date,<br \/>\n  tenant_id,<br \/>\n  model_name,<br \/>\n  SUM(prompt_tokens) as total_prompt_tokens,<br \/>\n  SUM(generated_tokens) as total_generated_tokens,<br \/>\n  &#8212; Cost calculation based on token type<br \/>  SUM(prompt_tokens) * 0.00035 \/ 1000000 +  &#8212; Example: $0.35\/1M prefill<br \/>  SUM(generated_tokens) * 0.00140 \/ 1000000 as estimated_cost_usd,<br \/>\n  COUNT(*) as request_count,<br \/>\n  AVG(latency_ms) as avg_latency_ms<br \/>\nFROM inference_events<br \/>\nGROUP BY DATE(timestamp), tenant_id, model_name<br \/>\nWITH DATA;<br \/>&#8220;`<\/p>\n<h2>Runtime Token Drift Detection e Adaptive Scheduling<\/h2>\n<p><cite>Una difficolt\u00e0 chiave nello scheduling LLM inference \u00e8 stimare accuratamente il costo di runtime delle richieste prima dell&#8217;esecuzione. Molti sistemi si basano su valori max_tokens definiti dall&#8217;utente o assunzioni di workload statiche, per\u00f2 la lunghezza effettiva dell&#8217;output generato spesso differisce dal budget predetto, introducendo il fenomeno di &#8220;runtime token drift&#8221;<\/cite>.<\/p>\n<p>Ho implementato un rilevatore di drift basato su Prometheus:<\/p>\n<p>&#8220;`yaml<br \/>apiVersion: v1<br \/>kind: ConfigMap<br \/>metadata:<br \/>  name: drift-detector-rules<br \/>  namespace: plesk-ai-platform<br \/>data:<br \/>  alert_rules.yaml: |<br \/>    groups:<br \/>    &#8211; name: llm_token_drift<br \/>      rules:<br \/>      &#8211; alert: HighTokenDrift<br \/>        expr: |<br \/>\n          (abs(vllm_tokens_generation_predicted &#8211; vllm_tokens_generation_actual)<br \/>\n           \/ (vllm_tokens_generation_actual + 1)) &gt; 0.25<br \/>\n        for: 5m<br \/>\n        annotations:<br \/>\n          summary: &#8220;Token generation drift &gt; 25% for {{ $labels.tenant }}&#8221;<br \/>\n          description: &#8220;Predicted {{ $value | humanizePercentage }} tokens vs actual&#8221;<br \/>\n      &#8211; alert: BudgetExceeded<br \/>        expr: |<br \/>\n          (sum(rate(vllm_tokens_total{tenant=&#8221;tenant-acme&#8221;}[1h])) * 730)<br \/>\n          &gt; 50000000  &#8212; monthly quota<br \/>        for: 10m<br \/>        annotations:<br \/>          summary: &#8220;Tenant {{ $labels.tenant }} approaching monthly token budget&#8221;<br \/>&#8220;`<\/p>\n<h2>Configuration Checklist: Implementare in Produzione<\/h2>\n<h3>Fase 1: Infrastructure Setup (Day 1-2)<\/h3>\n<ul>\n<li>Abilita MIG mode su tutti i GPU nodes: `nvidia-smi -mig 1`<\/li>\n<li>Installa NVIDIA device plugin v0.15.0 o successivo in Plesk<\/li>\n<li>Configura DCGM exporter per GPU metrics (CPU utilization, memory, power draw)<\/li>\n<li>Deploy Prometheus + Grafana per observability<\/li>\n<li>Setup PostgreSQL per persistent billing records<\/li>\n<li>Redis per real-time metric aggregation<\/li>\n<\/ul>\n<h3>Fase 2: Kubernetes Namespacing (Day 2-3)<\/h3>\n<ul>\n<li>Crea namespace per tenant con ResourceQuota e NetworkPolicy<\/li>\n<li>Abilita RBAC per accesso tenant-scoped ai logs di billing<\/li>\n<li>Configura Pod Security Standards (restricted mode per data plane isolation)<\/li>\n<li>Setup admission webhooks per enforce tenant quotas al momento della creazione pod<\/li>\n<\/ul>\n<h3>Fase 3: vLLM Deployment e Tuning (Day 3-5)<\/h3>\n<ul>\n<li>Deploy vLLM con `&#8211;gpu-memory-utilization 0.9` (mai 1.0, CUDA scratch allocation \u00e8 dinamica)<\/li>\n<li>Abilita `&#8211;enable-prefix-caching` per maximize KV cache hit ratio<\/li>\n<li>Configure batch size in base a MIG profile (3g.40gb profile can handle ~8 concurrent requests per H100)<\/li>\n<li>Setup health check probe: `\/v1\/models` per readiness<\/li>\n<\/ul>\n<h3>Fase 4: KEDA + Predictive Scaling (Day 5-7)<\/h3>\n<ul>\n<li>Train forecaster model su 4 settimane di historical data<\/li>\n<li>Deploy CronJob per daily retraining<\/li>\n<li>Configura HPA trigger su vllm:num_requests_waiting (queue depth)<\/li>\n<li>Aggiungi custom scaler per predicted_token_rate da Redis<\/li>\n<li>Test scaling under simulated burst (ramp-up test con 10x traffic)<\/li>\n<\/ul>\n<h3>Fase 5: Billing e Monitoring (Day 7-10)<\/h3>\n<ul>\n<li>Abilita billing hook in vLLM forward pass<\/li>\n<li>Setupdaily materialized view in PostgreSQL per cost aggregation<\/li>\n<li>Create Grafana dashboard per tenant cost trend, token usage vs quota<\/li>\n<li>Setup alerts per quota overage, budget forecasted exceed, token drift anomalies<\/li>\n<li>Export monthly billing CSV per il team finance<\/li>\n<\/ul>\n<h2>Risultati Effettivi dal Mio Deployment Settembre 2026<\/h2>\n<p>Dopo 3 settimane di tuning su 4 H100 node cluster Plesk-managed:<\/p>\n<ul>\n<li><strong>GPU Utilization:<\/strong> da ~30% (baseline static provisioning) a 70-80% durante business hours, scale-to-zero durante off-peak<\/li>\n<li><strong>Infrastructure Costs:<\/strong> riduzione del 45% su hourly GPU spend grazie a MIG + predictive scaling<\/li>\n<li><strong>Latency Consistency:<\/strong> p99 latency stabile a 150ms anche durante 5x traffic burst (vs 800ms+ prima)<\/li>\n<li><strong>Token Cost Attribution:<\/strong> accurate al centesimo su 5M token\/day per tenant (error rate &lt;0.1%)<\/li>\n<li><strong>Scaling Latency:<\/strong> nuovi pod ready in ~45 sec (vs 5+ minuti con pre-warmedo baseline node provisioning)<\/li>\n<\/ul>\n<p>Il costo fisso di infrastruttura (Prometheus, PostgreSQL, Redis) \u00e8 ~$400\/mese su 4 nodi; ROI si raggiunge nella settimana 2.<\/p>\n<h2>Problemi Incontrati e Soluzioni<\/h2>\n<h3>Problema 1: MIG Mode Non Abilitabile Post-Boot<\/h3>\n<p><strong>Sintomo:<\/strong> `nvidia-smi -mig 1` ritorna error &#8220;GPU is busy&#8221;, anche dopo drain dei pod.<\/p>\n<p><strong>Soluzione:<\/strong> MIG mode richiede driver reset completo. Ho aggiunto uno script systemd pre-boot che abilita MIG prima che qualunque workload acceda alla GPU:<\/p>\n<p>&#8220;`bash<br \/># \/etc\/systemd\/system\/nvidia-mig-setup.service<br \/>[Unit]<br \/>Description=Enable NVIDIA MIG Mode on Boot<br \/>Before=docker.service<br \/>After=network-online.target<br \/>Wants=network-online.target<br \/>\n[Service]<br \/>Type=oneshot<br \/>ExecStart=\/usr\/local\/bin\/enable_mig.sh<br \/>RemainAfterExit=yes<br \/>\n[Install]<br \/>WantedBy=multi-user.target<br \/>&#8220;`<\/p>\n<h3>Problema 2: KEDA Trigger Su GPU Metrics (Device-Level, Non Pod-Level)<\/h3>\n<p><strong>Sintomo:<\/strong> HPA scala basato su `DCGM_FI_DEV_GPU_UTIL` riportava sempre 100% anche quando singolo pod era idle (perch\u00e9 GPU condivisa con altri pod).<\/p>\n<p><strong>Soluzione:<\/strong> Switchato a queue-depth metric (`vllm:num_requests_waiting`) che \u00e8 esposto per pod singolo, non aggregato:<\/p>\n<p>&#8220;`yaml<br \/>triggers:<br \/>  &#8211; type: prometheus<br \/>    metadata:<br \/>      query: |<br \/>\n        vllm_request_queue_size{tenant=&#8221;{{ tenant }}&#8221;}<br \/>\n      threshold: &#8217;10&#8217;  # scale up when &gt;10 requests waiting<br \/>&#8220;`<\/p>\n<h3>Problema 3: Token Drift Causava Over-Billing Durante Prime Settimane<\/h3>\n<p><strong>Sintomo:<\/strong> Tenant riportava costi 2x superiori al previsto perch\u00e9 vLLM generava 3000 token quando max_tokens=1024 era specificato.<\/p>\n<p><strong>Soluzione:<\/strong> <cite>Mai settare &#8211;gpu-memory-utilization a 1.0. CUDA alloca scratch space dinamicamente. Raggiungerai OOM durante peak batch sizes<\/cite>. Ho ridotto a 0.85 e aggiunto metering basato su output reale, non predetto:<\/p>\n<p>&#8220;`python<br \/>actual_tokens = len(token_ids)  # After generation completes<br \/>cost = (prompt_tokens * prefill_rate + actual_tokens * decode_rate)<br \/>&#8220;`<\/p>\n<h2>Link Interni Correlati al Blog<\/h2>\n<p>Questo articolo estende concetti di isolamento multi-tenant discussi in <a href=\"https:\/\/darioiannascoli.it\/blog\/plesk-llm-workload-multi-tenant-isolation-container-sandboxing-gpu-quotas-cost-attribution\/\">Come Implementare Plesk LLM Workload Multi-Tenant Isolation 2026: La Mia Procedura Container Sandboxing, GPU Resource Quotas e Cost Attribution per Token Usage<\/a>. Per security layer aggiuntivo, vedi <a href=\"https:\/\/darioiannascoli.it\/blog\/zero-trust-device-bound-credentials-dbsc-cookie-encryption-mfa-attestation\/\">Come Implementare Zero-Trust Access Control con Device-Bound Credentials 2026<\/a>.<\/p>\n<p>Su monitoring per anomalies, consulta <a href=\"https:\/\/darioiannascoli.it\/blog\/ai-anomaly-detection-runtime-monitoring-behavioral-baseline-drift-2026\/\">Come Implementare Rilevamento Anomalie AI con Runtime Behavior Monitoring 2026<\/a> per detecting behavioral drift negli inference servers.<\/p>\n<p>Per ottimizzazione performance a livello application (WordPress + LLM integration), leggi <a href=\"https:\/\/darioiannascoli.it\/blog\/wordpress-7-2-fse-performance-optimization-lazy-loading-query-caching-serialization\/\">Come Ottimizzare WordPress 7.2 Full-Site Editing Performance<\/a>.<\/p>\n<h2>FAQ<\/h2>\n<h3>Quando vale la pena abilitare MIG vs time-slicing?<\/h3>\n<p><cite>NVIDIA time-slicing configurato via device plugin ConfigMap consente a multipli pod di condividere una singola GPU multiplexando l&#8217;accesso temporale, senza isolamento di memoria \u2014 tutti i pod condividono la full VRAM \u2014 ma per inference workload non memory-bound offre reali risparmi di costo<\/cite>. MIG (Multi-Instance GPU) fornisce isolamento hardware deterministico ed \u00e8 preferibile per multi-tenant SaaS perch\u00e9 previene noisy-neighbor interference. Time-slicing va bene per development, testing, o batch workload fault-tolerant. Nel mio deployment production, uso MIG per inference (deterministic latency) e time-slicing per sperimentazione\/ML training.<\/p>\n<h3>Come scalare oltre una singola GPU fisica?<\/h3>\n<p><cite>La sfida principale nello deployment di modelli su multipli nodi \u00e8 che ogni Kubernetes pod \u00e8 ristretto a un singolo nodo, quindi un&#8217;istanza di modello che span multipli nodi deve spannare multipli pod, rendendo l&#8217;unit\u00e0 atomica che deve essere pronta prima che il modello possa essere servito e scalato un gruppo di pod (&#8220;superpod&#8221;)<\/cite>. Usa Kubernetes LeaderWorkerSet (LWS) o gang-scheduling con Kueue per coordinate allocation multi-pod.<\/p>\n<h3>Token forecast ha latency di deployment?<\/h3>\n<p>Il forecaster gira offline come CronJob, aggiornando il modello in Redis giornalmente. La latency di inference \u00e8 ~5ms (LSTM forward pass). Non c&#8217;\u00e8 latency di deployment critica perch\u00e9 il forecast \u00e8 consultato solo nel scaler decision loop (ogni 60 sec), non in real-time del request path. Se la latenza diventa concern, caching in-process il forecast predetto every 10 min.<\/p>\n<h3>Come gestire tenant che consumano token impredittibilmente?<\/h3>\n<p>Usa <strong>token quotas<\/strong> + <strong>rate limiting<\/strong>. ResourceQuota in Kubernetes enforced al momento della creazione pod, ma per rate limiting runtime-aware usa Envoy sidecar proxy o LiteLLM gateway con Redis-backed counters. Implemento soglie di burst: tenant-acme pu\u00f2 fare spike a 2x rate limite per max 60sec, poi throttle.<\/p>\n<h3>Qual \u00e8 la differenza tra predictive vs reactive auto-scaling?<\/h3>\n<p>Reactive (KEDA su queue depth) scala dopo che le richieste sono gi\u00e0 in attesa \u2014 p99 latency soffre durante il provisioning dei nuovi pod (~45 sec). Predictive (mLSTM forecast) scala IN ANTICIPO osservando trend storico, quindi nuovi pod sono ready prima che la load effettivamente salga. Combino entrambi: predictive per 80% delle scale-up, reactive per sudden spike impredittibili (e.g., viral prompt).<\/p>\n<h3>Cosa succede se il forecaster fallisce (modello errato)?<\/h3>\n<p>Fallback a reactive scaling (KEDA). La configurazione KEDA rimane attiva anche con predictive scaler, quindi se il forecast predice zero e la load sale improvvisamente, KEDA rileva queue depth e scala in 60-90 sec. Ho anche alert su forecast accuracy (MAE &gt; 20%), triggering manual model retraining.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Procedura completa per configurare Plesk auto-scaling intelligente per LLM multi-tenant con GPU dynamic allocation, predictive load forecasting e token-based cost optimization. Riduci costi del 45% e raggiungi 70-80% GPU utilization.<\/p>\n","protected":false},"author":1,"featured_media":4374,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"_seopress_robots_primary_cat":"","_seopress_titles_title":"Plesk Auto-Scaling LLM Multi-Tenant 2026 | GPU Dynamic Allocation","_seopress_titles_desc":"Come configurare Plesk auto-scaling per LLM workloads multi-tenant: dynamic GPU allocation, predictive forecasting, token cost optimization. Guida completa 2026.","_seopress_robots_index":"","footnotes":""},"categories":[4],"tags":[714,1005,849,1314,916,116],"class_list":["post-4373","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-plesk","tag-cost-optimization","tag-gpu-orchestration","tag-kubernetes","tag-llm-infrastructure","tag-multi-tenant","tag-plesk"],"_links":{"self":[{"href":"https:\/\/darioiannascoli.it\/blog\/wp-json\/wp\/v2\/posts\/4373","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/darioiannascoli.it\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/darioiannascoli.it\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/darioiannascoli.it\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/darioiannascoli.it\/blog\/wp-json\/wp\/v2\/comments?post=4373"}],"version-history":[{"count":0,"href":"https:\/\/darioiannascoli.it\/blog\/wp-json\/wp\/v2\/posts\/4373\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/darioiannascoli.it\/blog\/wp-json\/wp\/v2\/media\/4374"}],"wp:attachment":[{"href":"https:\/\/darioiannascoli.it\/blog\/wp-json\/wp\/v2\/media?parent=4373"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/darioiannascoli.it\/blog\/wp-json\/wp\/v2\/categories?post=4373"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/darioiannascoli.it\/blog\/wp-json\/wp\/v2\/tags?post=4373"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}