Re: I/O Characterization - vSphere Environment

Many years ago I wrote an ugly perl script to analyze vscistats output files for all VMs on the hosts and present the overall average values, if that helps.

Start collecting data for all VMs/disks on the host, output all data into a file after you're done and stop:

# vscsiStats -s

# vscsiStats -p all > /tmp/vstats.txt

# vscsiStats -xr

Feed the files as argument to the script (you can use multiple files from other hosts as well to get an overall picture) and you will get an output like this:

./vscsistats.pl vstats1.txt vstats2.txt vstats3.txt

 

 

Avergae minimum values:

  latency      423.3

outstanding        1.0

      size      3697.8

 

 

Avergae maximum values:

  latency    35029.8

outstanding        29.1

      size    343608.9

 

 

Avergae mean values:

  latency      2139.6

outstanding        7.0

      size    48090.9

 

 

 

Total IO size distribution:

      Size        Count    Percent of all IOs

    <=4096        72618        62.85

    <=8192        13986        12.11

  <=32768          8635          7.47

  <=524288          6497          5.62

  <=16383          2999          2.60

  <=16384          2902          2.51

  <=49152          2759          2.39

  <=65535          1144          0.99

  <=131072          1110          0.96

  <=65536          1087          0.94

  <=262144          806          0.70

  <=81920          692          0.60

    <=2048          118          0.10

    <=1024            67          0.06

    <=4095            56          0.05

    <=512            46          0.04

    <=8191            12          0.01

  >524288            0          0.00

This is the script:

#!/usr/bin/env perl

 

 

use strict;

use warnings;

 

my @stats;

my (%min, %max, %mean, %iolength);

my $count = 0;

my $cur = 0;

my @temp;

 

foreach (@ARGV) {

  open my $fh, '<', $_ or die "Can't open file $!";

 

  while (<$fh>) {

    $cur++ if ($_ =~ /^\}$/);

    $stats[$cur] .= $_;

  }

  close $fh;

}

 

foreach(@stats) {

  $_ =~ m/(.+) worldGroupID.+min : (\d+).+max : (\d+).+mean : (\d+).+count : (\d+).+?\{(.+?)\}/sg;

  my @vals = ($2,$3,$4,$5,$6);

  if($vals[3] >= 1) { #filter zero IO count skewing values

    if ($1 =~ /IO lengths of commands for virtual machine/) {

      $min{'size'} += $vals[0];

      $max{'size'} += $vals[1];

      $mean{'size'} += $vals[2];

      $count++;

      push @temp, $vals[4];

    }

    elsif ($1 =~ /latency of IOs in Microseconds/) {

      $min{'latency'} += $vals[0];

      $max{'latency'} += $vals[1];

      $mean{'latency'} += $vals[2];

    }

    elsif ($1 =~ /number of outstanding IOs/) {

      $min{'outstanding'} += $vals[0];

      $max{'outstanding'} += $vals[1];

      $mean{'outstanding'} += $vals[2];

    }

}

}

 

print "\nAvergae minimum values:\n";

foreach(sort keys %min) { printf "%10s\t%8.1f\n", $_, ($min{$_} / $count); }

print "\nAvergae maximum values:\n";

foreach(sort keys %max) { printf "%10s\t%8.1f\n", $_, ($max{$_} / $count); }

print "\nAvergae mean values:\n";

foreach(sort keys %mean) { printf "%10s\t%8.1f\n", $_, ($mean{$_} / $count); }

 

my $totalios;

foreach(@temp) {

  foreach(split ('\n', $_)) {

    m/\s+(\d+)\s+\(([\<\>]\=?)\s+(\d+)\)/;

    if($3) {

      ($iolength{$2.$3} += $1);

      $totalios += $1;

    }

  }

}

 

print "\n\nTotal IO size distribution:\n";

printf "%10s\t%10s\t%10s\n", "Size", "Count", "Percent of all IOs";

foreach(sort { $iolength{$b} <=> $iolength{$a} } keys %iolength) {

  printf "%10s\t%10d\t%10.2f\n", $_, $iolength{$_}, (($iolength{$_} / $totalios) * 100 );

}