SaltSecurity.CollectorExtension 3.3.0

Salt Security Collector Extension

A comprehensive Azure App Service site extension that automatically captures all HTTP request and response traffic for security analysis without modifying your application code.

Features

• Complete Traffic Capture: Logs HTTP method, URL, headers, query strings, and body content
• Request & Response Logging: Captures both incoming requests and outgoing responses
• Detailed Timing Information: Measures and logs request duration
• Comprehensive Headers: Logs all request and response headers
• Body Content Capture: Captures request and response bodies (up to 10MB each)
• Daily Log Files: Automatically creates daily log files with timestamp-based naming
• Console Logging: Real-time logging to console for debugging
• Zero Code Changes: Works automatically via IHostingStartup
• Thread-Safe: Uses semaphore-based locking for concurrent request handling

Log File Location

All traffic logs are written to:

D:\home\LogFiles\SaltSecurityCollector\traffic_YYYYMMDD.log

Log Format

Each request/response pair is logged with the following information:

Request Section:

• Timestamp (UTC)
• HTTP Method
• Full URL (scheme, host, path, query string)
• Protocol version
• Content-Type and Content-Length
• All request headers
• Request body (if present)

Response Section:

• Timestamp (UTC)
• HTTP Status Code
• Duration (in milliseconds)
• Content-Type and Content-Length
• All response headers
• Response body (if present)

Sample Log Entry

========== REQUEST #1 ==========
Timestamp: 2024-11-18 10:30:45.123
Method: POST
URL: https://myapp.azurewebsites.net/api/users
Path: /api/users
QueryString: ?filter=active
Protocol: HTTP/1.1
ContentType: application/json
ContentLength: 156

Request Headers:
  Host: myapp.azurewebsites.net
  Content-Type: application/json
  Authorization: Bearer eyJ0eXAi...
  User-Agent: Mozilla/5.0

Request Body:
{"name":"John Doe","email":"john@example.com"}

========== RESPONSE #1 ==========
Timestamp: 2024-11-18 10:30:45.456
StatusCode: 201
Duration: 333.45ms
ContentType: application/json
ContentLength: 89

Response Headers:
  Content-Type: application/json
  Location: /api/users/12345
  Cache-Control: no-cache

Response Body:
{"id":"12345","name":"John Doe","email":"john@example.com","created":"2024-11-18T10:30:45Z"}

========== REQUEST #1 COMPLETE ==========

Console Logging

The extension also logs to the console (visible in Azure App Service logs) with the following markers:

• [SaltSecurity] - All extension-related log entries
• Request start/end markers
• Request counter for tracking
• Error messages if logging fails

Example console output:

[SaltSecurity] ========== REQUEST #1 START ==========
[SaltSecurity] Timestamp: 2024-11-18 10:30:45.123
[SaltSecurity] Method: POST
[SaltSecurity] Path: /api/users
[SaltSecurity] StatusCode: 201
[SaltSecurity] Duration: 333.45ms
[SaltSecurity] ========== REQUEST #1 COMPLETE ==========

Build

.\build.ps1

This will:

1. Build the extension in Release mode
2. Create the proper Azure Site Extension package structure
3. Copy all required files
4. Verify the package integrity
5. Display deployment instructions

Package Structure

The build script creates a package with the following structure matching the format in bkup->SaltSecurityCollectorExtension:

package/
├── applicationHost.xdt                      # IIS configuration transform
├── extension.xml                            # Extension metadata
├── Microsoft.Web.Xdt.Extensions.dll         # XDT transform support
├── additionalDeps/
│   └── shared/
│       └── Microsoft.AspNetCore.App/
│           └── 8.0.0/
│               └── SaltSecurityCollectorExtension.deps.json
└── store/
    └── x64/
        └── net8.0/
            └── saltsecuritycollectorextension/
                └── 1.0.0/
                    └── lib/
                        └── net8.0/
                            └── SaltSecurityCollectorExtension.dll

Deploy to Azure App Service via Kudu

Method 1: Manual Upload via Kudu Console

1. Build the extension: .\build.ps1
2. Navigate to your Kudu console: https://<your-app>.scm.azurewebsites.net/DebugConsole
3. Navigate to: C:\home\SiteExtensions
4. Create a new folder: SaltSecurityCollectorExtension
5. Upload all files from the package\ folder to this directory
   • You can drag and drop individual files
   • Or zip the contents and upload the zip (it will auto-extract)
6. Restart your app from Azure Portal
7. The extension will automatically start capturing traffic

Method 2: Using Kudu API

# Set your app details
$appName = "your-app-name"
$username = "deployment-username"  # Usually: $appName
$password = "deployment-password"  # Get from Azure Portal > Deployment Center

# Create zip from package contents
Compress-Archive -Path .\package\* -DestinationPath SaltSecurityCollectorExtension.zip -Force

# Upload via Kudu API
$pair = "$($username):$($password)"
$encodedCreds = [System.Convert]::ToBase64String([System.Text.Encoding]::ASCII.GetBytes($pair))
$headers = @{ Authorization = "Basic $encodedCreds" }

Invoke-RestMethod -Uri "https://$appName.scm.azurewebsites.net/api/zip/SiteExtensions/SaltSecurityCollectorExtension/" `
    -Method Put `
    -InFile "SaltSecurityCollectorExtension.zip" `
    -Headers $headers `
    -ContentType "application/zip"

Write-Host "Extension deployed! Restart your app to activate."

Verify Installation

After deployment and restart:

1. Check Extension Loading

   • Go to Kudu console: https://<your-app>.scm.azurewebsites.net
   • Navigate to: C:\home\SiteExtensions\SaltSecurityCollectorExtension
   • Verify all files are present

2. Check Environment Variables

   # In Kudu CMD console
   echo %ASPNETCORE_HOSTINGSTARTUPASSEMBLIES%
   # Should output: SaltSecurityCollectorExtension
   
   echo %DOTNET_ADDITIONAL_DEPS%
   # Should include: C:\home\SiteExtensions\SaltSecurityCollectorExtension\additionalDeps\
   
   echo %DOTNET_SHARED_STORE%
   # Should include: C:\home\SiteExtensions\SaltSecurityCollectorExtension\store
   

3. Check Console Logs

   • Azure Portal > Your App Service > Log Stream
   • Look for [SaltSecurity] markers
   • Should see "Extension Loading", "Middleware created" messages

4. Check Traffic Logs

   # In Kudu CMD console
   cd D:\home\LogFiles\SaltSecurityCollector
   dir
   # Should see: traffic_YYYYMMDD.log
   
   type traffic_20241118.log
   # Should see captured requests
   

Test the Extension

1. Make a request to your application:

   curl https://<your-app>.azurewebsites.net/
   

2. Check the log file:

   # In Kudu console
   cd D:\home\LogFiles\SaltSecurityCollector
   type traffic_20241118.log
   

3. Verify the request details are logged

Configuration

Custom Log Path

You can customize the log directory by setting an environment variable:

SALTSECURITY_LOG_PATH=D:\home\custom\path

Set this in Azure Portal > Configuration > Application Settings before deploying the extension.

How It Works

1. IHostingStartup: The extension uses the IHostingStartup interface to automatically inject itself into the ASP.NET Core pipeline when the application starts
2. Middleware Registration: The middleware is registered at the beginning of the request pipeline
3. Environment Variables: The applicationHost.xdt transform sets required environment variables:
   • ASPNETCORE_HOSTINGSTARTUPASSEMBLIES - Tells ASP.NET Core to load our extension
   • DOTNET_ADDITIONAL_DEPS - Points to our deps.json
   • DOTNET_SHARED_STORE - Points to our DLL location
4. Request Capture: Each request passes through the middleware which:
   • Captures request details (headers, body, timing)
   • Allows the request to process normally
   • Captures response details (headers, body, status)
   • Logs everything to the daily log file

Troubleshooting

Extension Not Loading

Check Environment Variables:

# In Kudu CMD
echo %ASPNETCORE_HOSTINGSTARTUPASSEMBLIES%

Check applicationHost.xdt was applied:

• The variables should point to the extension directory
• If not, the XDT transform may have failed

Solution:

• Re-upload the extension files
• Ensure Microsoft.Web.Xdt.Extensions.dll is present
• Restart the app

No Logs Being Created

Check log directory exists:

cd D:\home\LogFiles\SaltSecurityCollector

Check permissions:

• The extension creates the directory automatically
• Verify the app has write permissions to D:\home\LogFiles

Check console logs:

• Look for error messages with [SaltSecurity] prefix
• Errors during initialization will be logged to console

Extension Loaded But Not Capturing

Verify middleware is registered:

• Check console logs for "Middleware Instance Created"
• Check for "REQUEST #X START" messages

Check your app framework:

• Requires .NET 8.0 or compatible runtime
• Verify in Azure Portal > Configuration > General Settings

Performance Impact

The extension is designed for security analysis with minimal performance impact:

• Request/response bodies limited to 10MB each
• Thread-safe file writing with semaphore locking
• Asynchronous I/O operations
• For high-traffic production environments, monitor disk I/O and log file sizes

Requirements

• Azure App Service (Windows)
• .NET 8.0 or later runtime
• ASP.NET Core application

Version

Current Version: 1.0.0

Support

For issues, questions, or security concerns, contact Salt Security support.

License

Copyright © Salt Security. All rights reserved.