From: MATHEW,SHAJU \(HP-Roseville,ex1\) (shaju.mathew@hp.com)
Date: Wed Mar 19 2003 - 16:29:26 EST
Hello,
I have a question about the collection of network
metrics(specifically bytes_in and bytes_out) from the True64 DEC Alpha OSF1
platform. Since this is my first posting, forgive me if I'm not following
the accepted norms in any way.
We have some legacy software(an agent process that runs within a daemon
running as root) that collects performance data from a variety of sources.
For the OSF1 platform, we employ the technique of reading the kernel
memory(/dev/kmem), map it onto a suitable structure(ifnet) and proceed to
normalize the data. The agent is built on OSF1 v4.0(for the True64
platform), but distributed on later revisions also(for eg, on v5.0, v5.1,
etc).
Problem: These two network metric values show up as zero on v5.0 and v5.1
revisions of the OSF1 platform. On v4.0, the same source code(that reads
/dev/kmem) retrieves the correct data.
What is known so far: After I placed a call with True64 support, I found out
that the process we use(reading of the kernel memory) is not the supported
way to retrieve these metrics (specifically, there are the if_ibytes and
if_obytes members of the ifnet struct defined in /usr/include/net/if.h).
There has been kernel changes between 4.0 and 5.0 revisions of the OS, and
this caused this anomaly. At the bottom of this email, I'm attaching some
sample source code that isolates the part I'm referring to.
Question 1: Apparently, there're other supported APIs that retrieve this
same information(if_ibytes and if_obytes) on OSF1. Is anyone familiar with
mining this data? Do you have sample source? If this question is answered
satisfactorily, the next question will assume only academic interest.
Question 2: Below is an extract of an email reply from True64 support about
the kernel change and it's negative effect on legacy code:
>FYI, these statistics have been per-cpu for a while
>now for performance reasons, and are summed-up every
>so often, at least in V4.0X they were. In V5.0, they
>are only summed when you make an SIOCRDCTRS (read counters)
>ioctl request into the kernel.
I believe that this implies that the kernel memory reads would
continue to work on v5.0 provided I make a ioctl call prior to reading.
My question is about exactly how and where I need to make this call(ioctl).
Second parameter: SIOCRDCTRS
Third parameter: address of the ctrreq struct - I
initialized ctrreq.ctr_name with the name of the network card
What about the first parameter for ioctl? Has anyone done
this(triggering via ioctl)?
Thanks in advance!
Sample source code that displays BYTES_IN and BYTES_OUT:
Please look for a comment which reads "//PROBLEM". That
function(StrobeNetObj() employs the same code as the agent I'm
trouble-shooting.
I have attempted to exclude most of the irrevelant source code, but there's
still a lot of remnant structures that are of no interest. Please ignore
those. The only relevant functions are StrobeNetObj(), and it's execution
thread. This source will compile, but you need some proprietary headers(I
can supply those if you need 'em).
****************************************************************************
****************************************************************************
****************
/*
* Tru64 specific includes
*/
#include <mach.h> /* For struct task_info */
#include <net/if.h> /* For struct ifnet */
#include <sys/table.h> /* For table() */
#include <sys/sysinfo.h> /* For getsysinfo() */
#include <sys/mount.h>
#include <sys/user.h> /* For struct user */
#include <sys/procfs.h> /* For struct prpsinfo */
/* This is otherwise defined in machine/pmem.h */
#define PMEM_MANAGED 8
/*
* Some functions that do not have a header, or have other type
* of problems
*/
extern "C"
{
extern int getfsstat(struct statfs[], long, int);
extern int statvfs(const char *path, struct statvfs *buffer);
}
#include "coda_types.h"
static coda_UInt32_t sysobjseq = 0;
static coda_UInt32_t ticks = 0;
static coda_Real32_t fsutilpeak = 0;
typedef struct osf_vmstat
{
long memfree; /* Amount of free memory
*/
long membufcache; /* Amount of memory used by the buffercache
*/
long memusrunref; /* Amount of user unreferenced memory
*/
long pagefaults; /* Number of page faults
*/
long pageins; /* Number of page-ins
*/
long pageouts; /* Number of page-outs
*/
} osf_vmstat_t;
typedef struct osf_swapinfo
{
long swapsize; /* Amount of swap available in KB
*/
long swapfree; /* Amount of free swap in KB, reserved space
* excluded
*/
} osf_swapinfo_t;
typedef struct osf_pageinfo
{
long pageins; /* # of pages paged-in */
long pageouts; /* # of pages paged-out */
long pagescans; /* # of page scans */
long swapfree; /* Amount of free swap in KB counting
* reserved space as free
*/
} osf_pageinfo_t;
typedef struct osf_interrupts
{
long devintr; /* Device interrupts (non-clock) */
long context; /* Context switches */
long syscalls; /* Syscalls */
long forks; /* Number of forks */
long vforks; /* Number of vforks */
} osf_interrupts_t;
typedef struct
{
coda_UInt32_t boottime; /* Boot time */
coda_UInt32_t cpuclock; /* The cpu clock in Mhz */
coda_UInt64_t cpusystime; /* CPU system time */
coda_UInt64_t cpuusertime; /* CPU user time */
coda_UInt64_t cpuidletime; /* CPU idle time */
coda_Real32_t loadavg; /* Load average */
coda_UInt32_t memsize; /* Total memory size in KB */
coda_UInt32_t memused; /* Total memory used in KB */
coda_UInt32_t swapsize; /* Totap swap size in KB */
coda_UInt32_t swapused; /* Swap used in KB */
coda_UInt32_t pgin; /* Page-ins */
coda_UInt32_t pgout; /* Page-outs */
coda_UInt32_t pgscan; /* Page-scans */
coda_UInt32_t swapin; /* Swap-in in KB */
coda_UInt32_t swapout; /* Swap-out IN KB */
coda_UInt32_t numcpus; /* Number of CPUs */
coda_UInt32_t activecpus; /* Active number of CPUs */
coda_UInt32_t nnifs; /* Number of NICs */
coda_UInt32_t netpktsin; /* Network input packets */
coda_UInt32_t netpktsout; /* Netowrk output packets */
coda_UInt32_t netbytesin; /* Netowrk bytes received */
coda_UInt32_t netbytesout; /* Netowrk bytes sent */
coda_UInt32_t netcolls; /* Network collisions */
coda_UInt32_t neterrors; /* Netowrk errors */
coda_UInt32_t ndisks; /* Number of disks */
coda_UInt64_t diskio; /* Number of disk io */
coda_UInt64_t diskbytes; /* Number of disk bytes */
coda_UInt32_t startedprocs; /* # of started processes */
coda_UInt32_t aliveprocs; /* # of alive processes */
coda_UInt32_t activeprocs; /* # of active processes */
coda_UInt64_t interrupts; /* # of device interrupts */
coda_UInt64_t syscalls; /* # of system calls issued */
} global_t;
static global_t global;
struct fsystem
{
char *name; // filesystem name
int is_local; // is filesystem local?
};
// See /usr/include/sys/mount.h for a list of all supported filesystems
// ALERT: *******Check if this list is correct before release********
static fsystem dec_fstypes[] = {
{"None", 0},
{"UFS", 1},
{"NFS", 0},
{"MFS", 0},
{"PC", 0},
{"S5FS", 1},
{"CDFS", 0},
{"DFS", 0},
{"EFS", 1},
{"PROCFS", 0},
{"AdvFS", 1},
{"FFM", 0},
{"FDFS", 0},
{"ADDON", 0},
{"NFS3", 0}
};
// Converts a the short integer filesystem type to string
char *osf_getfsname(short ftype)
{
if (ftype < sizeof (dec_fstypes) / sizeof (dec_fstypes[0]))
return dec_fstypes[ftype].name;
return "Unknown";
}
typedef struct osf_fsinfo
{
char *mntdir; /* Filesystem mount point
*/
char *devname; /* Filesystem device
*/
char *fstype; /* Filesystem type
*/
int fslocal; /* True if filesystem is local
*/
struct statfs *fsbuf; /* Pointer to the buffer of statfs
structures */
int fscount; /* Total count of filesystems
*/
int fsidx; /* Index to next filesystem
*/
} osf_fsinfo_t;
typedef struct fsinfo
{
char *fsmntpoint; /* Filesystem mount point */
char *fsdevname; /* Filesystem device name */
char *fstype; /* Filesystem type */
dev_t devno; /* Device number */
} fsinfo_t;
fsinfo_t fsinf;
typedef unsigned long kaddr_t;
static struct nlist *ksym_nl = NULL; /* The array of kernel symbols
*/
static int ksym_initialized = 0; /* If symbols were already
initialized or not */
static int ksym_count = 0; /* Number of symbols
*/
static int ksym_alloc = 0; /* Number of slots allocated for symbols
*/
class KSym
{
private:
struct nlist *ksym_this;
void KSym_Init();
public:
KSym();
KSym(char *);
operator kaddr_t()
{
if (!ksym_initialized)
KSym_Init();
return (kaddr_t) (ksym_this->n_value);
}
};
KSym::KSym()
{
printf("ERROR!\n");
}
KSym::KSym(char *ksym_name)
{
/* Reallocate array if necessary */
if (ksym_alloc <= ksym_count) {
ksym_alloc += 16;
ksym_nl = (struct nlist *) realloc(ksym_nl,
(ksym_alloc +
1) * sizeof (struct nlist));
if (ksym_nl == NULL)
printf("ERROR!\n");
}
/* Add symbol name to array */
ksym_nl[ksym_count].n_name = ksym_name;
ksym_this = &ksym_nl[ksym_count];
ksym_count++;
}
void KSym::KSym_Init()
{
int retval;
int ii;
/* Pad last element */
ksym_nl[ksym_count].n_name = NULL;
retval = knlist(ksym_nl);
if (retval)
printf("ERROR!\n");
ksym_initialized = 1;
}
#define KSYM(x) KSym ksym_##x(#x)
/* /dev/kmem's filedescriptor */
static int kvm_fd = 0;
/*
* kvm_open: opens /dev/kmem for reading
* Return values: 0 on success, 1 on error
*/
int kvm_open(void)
{
printf("kvm_open()\n");
kvm_fd = open("/dev/kmem", O_RDONLY);
if (kvm_fd == -1) {
perror("kvm_open() failed");
return 1;
}
return 0;
}
/*
* kvm_read - reads kernel memory from address addr
* and stores its content to dest
* size is the number of bytes to copy
* Return values: 0 on success, 1 on error
*/
int kvm_read(kaddr_t addr, void *dest, int size)
{
//printf("kvm_read()\n");
int retval;
if (kvm_fd <= 0 && kvm_open()) {
printf("kvm_read() : null file descriptor\n");
return 1;
}
retval = lseek(kvm_fd, addr, SEEK_SET);
if (retval == -1) {
printf("kvm_read() : seek() failed\n");
perror("lseek() on /dev/kmem failed");
return 1;
}
retval = read(kvm_fd, dest, size);
if (retval < size) {
printf("kvm_read() : read() failed\n");
perror("read() from kernel memory failed");
return 1;
}
return 0;
}
/*
* The kernel symbol ifnet, the pointer to the first ifnet structure
*/
KSym ksym_ifnet("ifnet");
typedef struct osf_ifstat
{
char if_name[IFNAMSIZ]; /* Full interface name, eg. tu0 */
int if_unit; /* Instance number of device */
int if_inpackets; /* # of input packets */
int if_outpackets; /* # of output packets */
int if_inerror; /* # of errors in input */
int if_outerror; /* # of errors on output */
int if_collisions; /* # of collisions */
unsigned long if_inbytes; /* # of bytes received */
unsigned long if_outbytes; /* # of bytes sent */
kaddr_t if_next; /* Kernel address to next structure */
} osf_ifstat_t;
int osf_getifstat(kaddr_t idx, osf_ifstat_t * ifs)
{
printf("osf_getifstat()\n");
struct ifnet ifstat;
char ifname[IFNAMSIZ];
int retval;
retval = kvm_read((kaddr_t) idx, &ifstat, sizeof (ifstat));
if (retval) {
printf("osf_getifstat() : kvm_read() failed\n");
return 1;
}
retval = kvm_read((kaddr_t) ifstat.if_name, ifname, IFNAMSIZ);
if (retval) {
printf("osf_getifstat() : kvm_read() failed\n");
return 1;
}
printf("ifname is: %s\n", ifname);
/* Fill in structure */
sprintf(ifs->if_name, "%s%d", ifname, ifstat.if_unit); /* XXX
Security breach */
ifs->if_unit = ifstat.if_unit;
ifs->if_inerror = ifstat.if_ierrors;
ifs->if_outerror = ifstat.if_oerrors;
ifs->if_inbytes = ifstat.if_ibytes;
ifs->if_outbytes = ifstat.if_obytes;
ifs->if_inpackets = ifstat.if_ipackets;
ifs->if_outpackets = ifstat.if_opackets;
ifs->if_collisions = ifstat.if_collisions;
ifs->if_next = (kaddr_t) ifstat.if_next;
//printf("Net in bytes: %d; ", ifstat.if_ibytes);
//printf("Net out bytes: %d\n", ifstat.if_obytes);
//printf("Net in packets: %d;", ifstat.if_ipackets);
//printf("Net out packets: %d\n", ifstat.if_opackets);
return 0;
}
int osf_ifstat_open(osf_ifstat_t * ifs)
{
printf("osf_ifstat_open()\n");
kaddr_t addr;
int retval;
retval = kvm_read(ksym_ifnet, &addr, sizeof (addr));
if (retval)
return 1;
ifs->if_next = addr;
return 0;
}
int osf_ifstat_read(osf_ifstat_t * ifs)
{
return ifs->if_next ? osf_getifstat(ifs->if_next, ifs) : 1;
}
typedef struct
{
char *name;
coda_UInt32_t id;
coda_UInt32_t pktsin;
coda_UInt32_t pktsout;
coda_UInt32_t bytesin;
coda_UInt32_t bytesout;
coda_UInt32_t errorsin;
coda_UInt32_t errorsout;
coda_UInt32_t collisions;
} netdata_t;
static netdata_t ndat;
void StrobeNetObj()
{
printf("StrobeNetObj()\n");
osf_ifstat_t ifstat;
int ifcnt;
/* Reset global statistics */
global.netpktsin = 0;
global.netpktsout = 0;
global.netbytesin = 0;
global.netbytesout = 0;
global.netcolls = 0;
global.neterrors = 0;
ifcnt = 0;
for (osf_ifstat_open(&ifstat); !osf_ifstat_read(&ifstat); ) {
printf("Network card id is: %s\n", ifstat.if_name);
static const char lo_str[] = "lo";
if (strncmp(lo_str, ifstat.if_name, sizeof lo_str - 1) == 0) {
continue;
}
ndat.id = ifcnt++;
ndat.name = ifstat.if_name;
ndat.pktsin = ifstat.if_inpackets;
ndat.pktsout = ifstat.if_outpackets;
//printf("IN PACKETS: %u;", ifstat.if_inpackets);
//printf("OUT PACKETS: %u\n", ifstat.if_outpackets);
ndat.bytesin = ifstat.if_inbytes;
ndat.bytesout = ifstat.if_outbytes;
// PROBLEM
printf("BYTES_IN : %u;", ifstat.if_inbytes);
printf("BYTES_OUT: %u\n", ifstat.if_outbytes);
ndat.errorsin = ifstat.if_inerror;
ndat.errorsout = ifstat.if_outerror;
ndat.collisions = ifstat.if_collisions;
global.netpktsin += ifstat.if_inpackets;
global.netpktsout += ifstat.if_outpackets;
global.netbytesin += ifstat.if_inbytes;
global.netbytesout += ifstat.if_outbytes;
global.netcolls += ifstat.if_collisions;
global.neterrors += ifstat.if_inerror + ifstat.if_outerror;
}
global.nnifs = ifcnt;
printf("GLOBAL IN PACKETS: %u\n", global.netpktsin);
printf("GLOBAL OUT PACKETS: %u\n", global.netpktsout);
}
typedef struct osf_procuarea
{
long starttime; /* Process start time
*/
ssize_t datasize; /* Process data segment size
*/
ssize_t stacksize; /* Process stack segment size
*/
long pcpuuser; /* Number of CPU seconds spent in user space
*/
long pcpusys; /* Number of CPU seconds spent in kernel
space*/
long minorfaults; /* Minor faults
*/
long majorfaults; /* Major faults
*/
long blockin; /* # of blockin operations
*/
long blockout; /* # of blockout operations
*/
} osf_procuarea_t;
typedef struct osf_procinfo
{
uid_t uid; /* Process UserID */
uid_t euid; /* Effective process UID */
gid_t gid; /* Group ID */
pid_t pid; /* Process ID */
pid_t ppid; /* Parent Process ID */
int pgrp; /* Process group */
char procname[FNAME_SZ]; /* Process name + command line */
int ttydev; /* Controlling terminal number */
int stop_reason; /* Stop reason */
long vss; /* Virtual memory size in KB */
long rss; /* Resident memory size in KB */
long nthreads; /* Number of threads */
long ctime; /* Sum of children's U/S time */
DIR *procdir; /* /proc directory entry */
} osf_procinfo_t;
typedef struct
{
pid_t pid;
pid_t ppid;
uid_t uid;
uid_t euid;
gid_t gid;
char *pname;
char state;
coda_UInt64_t minflt;
coda_UInt64_t majflt;
coda_UInt64_t utime;
coda_UInt64_t stime;
coda_Int32_t priority;
coda_Int32_t nice;
coda_UInt64_t starttime;
coda_UInt64_t vss;
coda_UInt64_t rss;
coda_UInt32_t nthreads;
} procdata_t;
static procdata_t pdat;
int main()
{
StrobeNetObj();
printf("All done\n");
}
********************************************
Shaju Mathew
Software Design Engineer
Performance Technology Center
OpenView Software R & D
Hewlett-Packard Company
California 95747, USA
Ph: 916-785-9018
Shaju.Mathew@hp.com
shajum@hpptc51.rose.hp.com
********************************************
This archive was generated by hypermail 2.1.7 : Sat Apr 12 2008 - 10:49:12 EDT