2023.11.26

VERSION

@@ -1,3 +1,2 @@
-February 08 2023
-
+November 26 2023
 

doc/CHANGES

@@ -15079,3 +15079,60 @@ February 08 2023
 midi2abc, mftext, midistats: modified midifile.c so that it indicates where
 it encounters an unexpected byte in the input midi file.
 
+
+March 24 2023
+More functions for the percussion track was added to midistats.c to
+support some future analysis.
+
+
+June 25 2023
+Midistats runs incorrectly if the midi file has no program channel
+commands. When a program channel command is missing, the program should
+be the acoutic piano (0) by default.
+
+If more than one channel with the same program color, they should be
+added together.
+
+
+August 22 2023
+
+Midistats improved the calculation of the number of rhythm patterns
+in a channel (barChn[chan].rhythmPattern) by allowing for dithering
+of note onset. Introduced dithermargin variable in function
+stats_noteon(). In function, output_track_summary(), suppressed
+notemeanpitch for percussion channel.
+
+
+October 25 2023
+
+Midistats returns track activity (note on/off) for every track.
+
+
+November 1 2023
+
+Midistats returns the control volume settings for every track,
+identifies midi files whose note timings are not quantized.
+
+
+November 2 2023
+
+abc2midi bug: In the following example, not all notes are
+tied correctly.
+
+X:1
+T: tied note
+M: 2/4
+L: 1/4
+K: C
+D2-|:D2 |[1CD-:|[2CD|
+
+No fix is available.
+
+
+November 8 2023
+
+midistats: extended the size of arrays (midievents and pulsecounter) to
+handle certain midi files. The function stats_interpret_pulseCounter()
+can detect midi files containing triplets and nonquantized notes.
+
+midicopy: extended to handle midi files with up to 150 tracks.

doc/drums.txt

@@ -0,0 +1,86 @@
+Advamced Percussion Analysis
+in the Midistats Program
+
+This is an addendum to the midistats.1 file.
+
+The MIDI file devotes channel 9 to the percussion instruments
+and over 60 percussion instruments are defined in the MIDI
+standard. Though there is a lot of diversity in the percussion
+track, for most MIDI files only the first 10 or so percussion
+instruments are important in defining the character of the track. The
+program Midiexplorer has various tools for exposing the percussion
+channel which are described in the documentation. The goal
+here is to find the essential characteristics of the percussion
+track which distinguishes the MIDI files. This is attempted
+in the program midistats.  Here is a short description.
+
+-corestats
+Produces a line with 5 numbers separated by tabs. eg
+1	8	384	4057	375
+It returns the number of tracks, the number of channels, the
+number of divisions per quarter note beat (ppqn),
+the number of note onsets in the midi file, and the maximum
+number of quarter note beats in midi file.
+
+-pulseanalysis
+Counts the number of note onsets as a function of its onset time
+relative to a beat, grouping them into 12 intervals and returns
+the result as a discrete probability density function. Generally,
+the distribution consists of a couple of peaks corresponding
+to quarter notes or eigth notes. If the distribution is flat,
+it indicates that the times of the note occurrences have not been
+quantized into beats and fractions. Here is a sample output.
+0.3496,0.0000,0.0000,0.1602,0.0000,0.0002,0.2983,0.0000,0.0000,0.1914,0.0002,0.0001
+
+-panal
+Counts the number of note onsets for each percussion instrument. The first
+number is the code (pitch) of the instrument, the second number is the
+number of occurrences. eg.
+35 337	37 16	38 432	39 208	40 231	42 1088	46 384	49 42	54 1104	57 5	70 1040	85 16	
+
+-ppatfor n
+where n is the code number of the percussion instrument. Each beat
+is represented by a 4 bit number where the position of the on-bit
+indicates the time in the beat when the drum onset occurs. The bits
+are ordered from left to right (higher order bits to lower order
+bits). This is the order of bits that you would expect in a
+time series.
+Thus 0 indicates that there was no note onset in that beat, 1 indicates
+a note onset at the end of the beat, 4 indicates a note onset
+in the middle of the beat, and etc. The function returns a string
+of numbers ranging from 0 to 7 indicating the presence of note onsets
+for the selected percussion instrument for the sequence of beats
+in the midi file. Here is a truncated sample of the output.
+
+0 0 0 0 0 0 0 0 1 0 0 4 1 0 0 4 1 0 0 4 1 0 0 4 1 0 0 4 1 0 0 4 1 4 4 0
+1 0 0 0 1 0 5 0 1 0 5 0 1 0 5 0 1 0 5 0 1 0 5 0 1 0 5 0 1 0 5 0 1 0 0 0
+1 0 5 0 1 0 5 0 1 etc.
+
+One can see a repeating 4 beat pattern.
+
+-ppat
+midistats attempts to find two percussion instruments in the midi file
+which come closest to acting as the bass drum and snare drum.
+If it is unsuccessful, it returns a message of its failue. Otherwise,
+encodes the position of these drum onsets in a 8 bit byte for each
+quarter note beat in the midi file. The lower (right) 4 bits encode the
+bass drum and the higher (left) 4 bits encode the snare drum in the
+same manner as described above for -ppatfor.
+0 0 0 0 0 0 0 0 0 0 33 145 33 145 33 145 33 145 33 145 33 145 33 145 33 145
+33 145 33 145 33 145 33 145 33 145 33 145 33 145 33 145 33 145 33 145 33 145
+33 145 33 145 33 145 33 145 33 145 33 and etc.
+
+-ppathist
+computes and displays the histogram of the values that would appear
+when running the -ppat. eg.
+bass 35 337
+snare 38 432
+1 (0.1) 64  32 (2.0) 8  33 (2.1) 136  144 (9.0) 8  145 (9.1) 136 
+
+The bass percussion code, the number of onsets, and the snare
+percussion code and the number of onsets are given in the
+first two lines. In the next line the number of occurrences of
+each value in the -ppat listing is given. The number in parentheses
+splits the two 4-bit values with a period. Thus 33 = (2*16 + 1).
+
+

doc/midistats.1

@@ -1,4 +1,4 @@
-.TH MIDISTATS 1 "9 December 2022"
+.TH MIDISTATS 1 "17 November 2023"
 .SH NAME
 \fBmidistats\fP \- program to summarize the statistical properties of a midi file
 .SH SYNOPSIS
@@ -36,7 +36,14 @@ the sum of the MIDI pitches for all the notes,
 the sum of the note durations in MIDI pulse units,
 the number of control parameter messages,
 the number of pressure messages.
-and the number of distinct rhythm patterns for each channel
+the number of distinct rhythm patterns for each channel
+the number of pulses the channel was inactive
+the minimum pitch value
+the maximum pitch value
+the minimum note length in pulses
+the maximum note length in pulses
+the number of gaps in the channel
+the entropy of the pitch class histogram for that channel
 .PP
 After processing all the individual tracks, the following information
 applies to the entire midi file.
@@ -70,9 +77,9 @@ that occur in the midi file.
 pitchact is a similar histogram but is weighted by the length of
 the notes.
 .PP
-quietTime is used to compute the track/channel spread in midiexplorer.
-It is computed by summing up all the midi pulses which occur
-in gaps greater than 8 beats.
+chnact returns the amount of note activity in each channel.
+.PP
+trkact returns the number of notes in each track.
 .PP
 totalrhythmpatterns is the total number of bar rhythm patterns for
 all channels except the percussion channel.
@@ -80,6 +87,25 @@ all channels except the percussion channel.
 collisions. Midistats counts the bar rhythm patterns using a hashing
 function. Presently collisions are ignored so occasionally two
 distinct rhythm patterns are counted as one.
+.SH Advance Percussion Analysis Tools
+.PP
+A number of experimental tools for analyzing the percussion channel
+(track) were introduced into midistats and are accessible through
+the runtime arguments. When these tools are used in a script which
+runs through a collection of midi files, you can build a database
+of percussion descriptors. Some more details are given in the
+file drums.txt which comes with this documentation.
+
+.SH OPTIONS
+.TP
+.B -corestats
+.TP
+.B -pulseanalysis
+.TP
+.B etc. (See drums.txt in doc folder.)
+
+
+
 .SH AUTHOR
 Seymour Shlien <fy733@ncf.ca>
 

doc/readme.txt

@@ -5,21 +5,19 @@ abc2midi version 4.84 January 06 2023
 abc2abc  version 2.20 February 07 2023
 yaps     version 1.92 January 06 2023
 abcmatch version 1.82 June 14 2022
-midicopy version 1.38 May 06 2022
-midistats version 0.59 February 08 2023
+midicopy version 1.39 November 08 2022
+midistats version 0.81 November 26 2023
 
 24th January 2002
-
 Copyright James Allwright
-J.R.Allwright@westminster.ac.uk
+jamesallwright@yahoo.co.uk
 University of Westminster,
 London, UK
 
-October 2021
-
-Seymour Shlien
-Ottawa, Canada
+August 2023
+Copyright Seymour Shlien
 fy733@ncf.ca
+Ottawa, Canada
 
 This is free software. You may copy and re-distribute it under the terms of 
 the GNU General Public License version 2 or later, which is available from

midicopy.c

@@ -52,7 +52,7 @@
 
 
 
-#define VERSION "1.38 May 05 2022 midicopy"
+#define VERSION "1.39 November 07 2023 midicopy"
 #include "midicopy.h"
 #define NULLFUNC 0
 #define NULL 0
@@ -96,8 +96,9 @@ long max_currtime = 0;
 long Mf_currcopytime = 0L;	/* time of last copied event */
 char *trackdata = NULL;
 long trackdata_length, trackdata_size;
-char *trackstr[64]; /* [SS] 2017-10-20  2019-07-05*/
-int trackstr_length[64]; /* [SS] 2017-10-20 2019-07-05*/
+/* char *trackstr[64];  [SS] 2017-10-20  2019-07-05*/
+char *trackstr[150]; /* [SS] 2023-11-07 */
+int trackstr_length[150]; /* [SS] 2017-10-20 2019-07-05* 2023-11-07*/
 int trkid = 0;
 int activetrack;
 int nochanmsg = 1;
@@ -1290,7 +1291,7 @@ build_new_midi_file (format, ntracks, division, fp)
 
   get_tempo_info_from_track_1 ();
 
-  if (ntracks > 63) {printf("too many tracks\n"); exit(1); }
+  if (ntracks > 149) {printf("too many tracks\n"); exit(1); }
 
   /* The rest of the file is a series of tracks */
   for (i = 0; i < ntracks; i++)
@@ -1836,7 +1837,7 @@ main (int argc, char *argv[])
       printf ("-ver  version information\n");
       printf ("-trks n1,n2,..(starting from 1)\n");
       printf ("-xtrks n1,n2,.. (tracks to exclude)\n"); /* [SS] 2013-10-27 */
-      printf ("-xchns n1,n2,.. (tracks to exclude)\n"); /* [SS] 2017-12-06 */
+      printf ("-xchns n1,n2,.. (channels to exclude)\n"); /* [SS] 2022-11-12 */
       printf ("-chns n1,n2,..(starting from 1)\n");
       printf ("-from n (in midi ticks)\n");
       printf ("-to n   (in midi ticks)\n");

midistats.c

@@ -18,7 +18,7 @@
  * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA  02110-1301, USA
  */
  
-#define VERSION "0.62 March 08 2023 midistats"
+#define VERSION "0.81 November 26 2023 midistats"
 
 #include <limits.h>
 /* Microsoft Visual C++ Version 6.0 or higher */
@@ -49,10 +49,11 @@ extern char* strchr();
 #include "midifile.h"
 void initfuncs();
 void stats_finish();
-float histogram_entropy (int *histogram, int size); 
+float histogram_perplexity (int *histogram, int size); 
 void stats_noteoff(int chan,int pitch,int vol);
 void stats_eot ();
-
+#define max(a,b)  (( a > b ? a : b))
+#define min(a,b) (( a < b ? a : b))
 
 /* Global variables and structures */
 
@@ -62,9 +63,11 @@ static FILE *F;
 static FILE *outhandle; /* for producing the abc file */
 
 int tracknum=0;  /* track number */
+int lasttrack = 0; /* lasttrack */
 int division;    /* pulses per quarter note defined in MIDI header    */
 int quietLimit;  /* minimum number of pulses with no activity */
 long tempo = 500000; /* the default tempo is 120 quarter notes/minute */
+int bpm = 120; /*default tempo */
 long laston = 0; /* length of MIDI track in pulses or ticks           */
 int key[12];
 int sharps;
@@ -75,6 +78,9 @@ int debug;
 int pulseanalysis;
 int percanalysis;
 int percpattern;
+int percpatternfor;
+int percpatternhist;
+int pitchclassanalysis;
 int corestats;
 int chordthreshold; /* number of maximum number of pulses separating note */
 int beatsPerBar = 4; /* 4/4 time */
@@ -82,13 +88,14 @@ int divisionsPerBar;
 int unitDivision;
 int maximumPulse;
 int lastBeat;
+int hasLyrics = 0;
 
 
 struct eventstruc {int onsetTime;
                    unsigned char channel;
 		   unsigned char pitch;
 		   unsigned char velocity;
-                   ;} midievents[40000];
+                   ;} midievents[50000];
 
 int lastEvent = 0;
 
@@ -100,6 +107,9 @@ int stats = 0; /* flag - gather and print statistics            */
 int pulseanalysis = 0;
 int percanalysis = 0;
 int percpattern = 0;
+int percpatternfor = 0;
+int percpatternhist = 0;
+int pitchclassanalysis = 0;
 int corestats = 0;
 
 
@@ -108,9 +118,14 @@ int trackcount = 0;
 
 int notechan[2048],notechanvol[2048]; /*for linking on and off midi
 					channel commands            */
-int last_tick[17]; /* for getting last pulse number in MIDI file */
+int lastTick[2048]; /* for getting last pulse number for chan (0-15) and pitch (0-127) in MIDI file */
 int last_on_tick[17]; /* for detecting chords [SS] 2019-08-02 */
+int channel_active[17]; /* for dealing with chords [SS] 2023-08-30 */
+int channel_used_in_track[17]; /* for dealing with quietTime [SS] 2023-09-06 */
 
+int histogram[256];
+unsigned char drumpat[8000];
+int percnum;
 
 
 
@@ -122,7 +137,11 @@ struct trkstat {
   int notecount[17];
   int chordcount[17];
   int notemeanpitch[17];
+  int notepitchmin[17];
+  int notepitchmax[17];
   int notelength[17];
+  int notelengthmin[17];
+  int notelengthmax[17];
   int pitchbend[17];
   int pressure[17];
   int cntlparam[17];
@@ -132,6 +151,9 @@ struct trkstat {
   int lastNoteOff[17];
   int quietTime[17];
   int rhythmpatterns[17];
+  int numberOfGaps[17];
+  int chanvol[17];
+  float pitchEntropy[17];
   } trkdata;
 
 /* The trkstat references the individual channels in the midi file.
@@ -153,8 +175,10 @@ int pitchhistogram[12]; /* pitch distribution for non drum notes */
 int channel2prog[17]; /* maps channel to program */
 int channel2nnotes[17]; /*maps channel to note count */
 int chnactivity[17]; /* [SS] 2018-02-02 */
+int trkactivity[40]; /* [SS] 2023-10-25 */
 int progactivity[128]; /* [SS] 2018-02-02 */
 int pitchclass_activity[12]; /* [SS] 2018-02-02 */
+int chanpitchhistogram[204]; /* [SS] 2023-09-13 */
 
 
 /* [SS] 2017-11-01 */
@@ -177,7 +201,7 @@ static int progmapper[] = {
 16, 16, 16, 16, 16, 16, 16, 16
 };
 
-int pulseCounter[480];
+int pulseCounter[1024];
 int pulseDistribution[24];
 
 struct barPattern {
@@ -398,6 +422,7 @@ void stats_header (int format, int ntrks, int ldivision)
   quietLimit = ldivision*8;
   divisionsPerBar = division*beatsPerBar;
   unitDivision = divisionsPerBar/24;
+  lasttrack = ntrks; /* [SS] 2023-10-25 */
   printf("ntrks %d\n",ntrks);
   printf("ppqn %d\n",ldivision);
   chordthreshold = ldivision/16; /* [SS] 2018-01-21 */
@@ -410,8 +435,10 @@ void stats_header (int format, int ntrks, int ldivision)
     trkdata.cntlparam[i] = 0; /* [SS] 2022-03-04 */
     trkdata.pressure[i] = 0; /* [SS] 2022-03-04 */
     trkdata.quietTime[i] = 0; /* [SS] 2022-08-22 */
+    trkdata.numberOfGaps[i] = 0; /* [SS] 2023-09-07 */
+    trkdata.chanvol[i] = 0; /* [SS] 2023-10-30 */
     progcolor[i] = 0;
-    channel2prog[i] = -1;
+    channel2prog[i] = 0; /* [SS] 2023-06-25-8/
     channel2nnotes[i] = 0;
     chnactivity[i] = 0; /* [SS] 2018-02-02 */
     }
@@ -419,6 +446,7 @@ void stats_header (int format, int ntrks, int ldivision)
   for (i=0;i<12;i++) pitchhistogram[i] = 0; /* [SS] 2017-11-01 */
   for (i=0;i<12;i++) pitchclass_activity[i] = 0; /* [SS] 2018-02-02 */
   for (i=0;i<128;i++) progactivity[i] = 0; /* [SS] 2018-02-02 */
+  for (i=0;i<40;i++) trkactivity[i]=0; /* [SS] 2023-10-25 */
 }
 
 void determine_progcolor ()
@@ -445,6 +473,61 @@ int i;
   }
 
 
+/* [SS] 2023-10-30 */
+void stats_interpret_pulseCounter () {
+int i,j;
+int maxcount,ncounts,npeaks,npositives,peaklimit;
+int maxloc;
+float threshold,peak;
+int decimate;
+float tripletsCriterion8,tripletsCriterion4;
+int resolution = 12;
+int nzeros;
+threshold = 10.0/(float) division;
+maxcount = 0;
+ncounts = 0;
+npeaks = 0;
+for (i=0;i<division;i++) {
+  ncounts = ncounts + pulseCounter[i];
+  if (pulseCounter[i] > maxcount) {
+       maxloc = i;
+       maxcount = pulseCounter[i];
+       } 
+  } 
+peaklimit = (int) (ncounts * 0.020);
+for (i=0;i<division;i++) {
+  if (pulseCounter[i] > peaklimit) npeaks++;
+  }
+for (i = 0; i < resolution; i++) pulseDistribution[i] = 0;
+decimate = division/resolution;
+for (i = 0; i < division; i++) {
+   j = i/decimate;
+   pulseDistribution[j] += pulseCounter[i];
+  }
+
+/* count zeros */
+nzeros = 0;
+for (i=0;i<resolution;i++) if((float) pulseDistribution[i]/(float) ncounts < 0.015 ) nzeros++;
+npositives = resolution - nzeros;
+if (nzeros > 3 && (float) pulseDistribution[resolution-1]/(float) ncounts < 0.1) {printf("clean_quantization\n");
+ } else if ((float) pulseDistribution[resolution-1]/(float) ncounts > 0.09 ||
+            npeaks > npositives) {printf("dithered_quantization\n");
+ } else {
+peak = (float) maxcount/ (float) ncounts;
+if (peak < threshold)  printf("unquantized\n");
+}
+
+tripletsCriterion8 = (float) pulseDistribution[8]/ (float) ncounts;
+tripletsCriterion4 = (float) pulseDistribution[4]/ (float) ncounts;
+if (tripletsCriterion8 > 0.10 || tripletsCriterion4 > 0.10) printf("triplets\n");
+if (pulseDistribution[0]/(float) ncounts > 0.95) printf("qnotes");
+/*
+printf("pulseDistribution:");
+for (i=0;i<resolution;i++) printf("%6.3f",(float) pulseDistribution[i]/(float) ncounts);
+printf("\n");
+printf("nzeros = %d npeaks = %d \n",nzeros,npeaks);
+*/
+}
 
 void stats_finish()
 {
@@ -474,7 +557,7 @@ if (nprogs > 0) output_progs_data();
 else {
   for (i=0;i<17;i++) 
     if(chnactivity[i] > 0) 
-       progactivity[channel2prog[i]] = chnactivity[i];
+       progactivity[channel2prog[i]] += chnactivity[i]; /* [SS] 2023-06-25 */
   output_progs_data();
   }
 
@@ -502,38 +585,39 @@ if (npulses > 0)
   for (i=0;i<12;i++) printf("%5.2f ",pitchclass_activity[i]/(double) npulses);
 else 
   for (i=0;i<12;i++) printf("%5.2f ",(double) pitchclass_activity[i]);
+printf("\nchanvol "); /* [SS] 2023-10-30 */
+for (i=1;i<17;i++) printf("%4d ",trkdata.chanvol[i]);
 printf("\nchnact "); /* [SS] 2018-02-08 */
 if (npulses > 0)
-  for (i=1;i<17;i++) printf("%5.2f ",chnactivity[i]/(double) trkdata.npulses[0]);
+  for (i=1;i<17;i++) printf("%5.3f ",chnactivity[i]/(double) trkdata.npulses[0]);
 else 
-  for (i=0;i<17;i++) printf("%5.2f ",(double) chnactivity[i]);
-printf("\nquietTime ");
-for (i=1;i<17;i++) {
-  delta = trkdata.npulses[0] - trkdata.quietTime[i];
-  if (trkdata.quietTime[i] < quietLimit) delta = 0;
-  delta = delta / (double) trkdata.npulses[0];
-  /* printf (" %5.3f ", delta); */
-  printf (" %d ", trkdata.quietTime[i]);
-  }
-
-printf("\npitchentropy %f\n",histogram_entropy(pitchclass_activity,12));
+  for (i=0;i<17;i++) printf("%5.3f ",(double) chnactivity[i]);
+printf("\ntrkact ");
+  lasttrack++;
+  for (i=0;i<lasttrack;i++) printf("% 5d",trkactivity[i]);
+printf("\npitchperplexity %f\n",histogram_perplexity(pitchclass_activity,12));
 printf("totalrhythmpatterns =%d\n",nrpatterns);
 printf("collisions = %d\n",ncollisions);
+if (hasLyrics) printf("Lyrics\n");
+stats_interpret_pulseCounter ();
 printf("\n");
 }
 
 
 
-float histogram_entropy (int *histogram, int size) 
+float histogram_perplexity (int *histogram, int size) 
   {
+  /* The perplexity is 2 to the power of the entropy */
   int i;
   int total;
   float entropy;
   float e,p;
   total = 0;
   entropy = 0.0;
+  //printf("\nhistogram_entropy of:");
   for (i=0;i<size;i++) {
     total += histogram[i];
+    //printf(" %d",histogram[i]);
     } 
   for (i=0;i<size;i++) {
     if (histogram[i] < 1) continue;
@@ -541,7 +625,8 @@ float histogram_entropy (int *histogram, int size)
     e = p*log(p);
     entropy = entropy + e;
     } 
-  return -entropy/log(2.0);
+  //printf("\n");
+  return pow(2.0,-entropy/log(2.0));
   } 
 
 
@@ -575,10 +660,18 @@ for (i=1;i<17;i++) {
    printf("trkinfo ");
    printf("%d %d ",i,trkdata.program[i]); /* channel number and program*/
    printf("%d %d ",trkdata.notecount[i],trkdata.chordcount[i]);
-   printf("%d %d ",trkdata.notemeanpitch[i], trkdata.notelength[i]);
+   /* [SS] 2023-08-22 */
+   if (i != 10) printf("%d %d ",trkdata.notemeanpitch[i], trkdata.notelength[i]);
+   else 
+     printf("-1  0 ");
    printf("%d %d ",trkdata.cntlparam[i],trkdata.pressure[i]); /* [SS] 2022-03-04 */
-   printf("%d %d",trkdata.quietTime[i],trkdata.rhythmpatterns[i]);
-   trkdata.quietTime[i] = 0;
+   printf("%d %d ",trkdata.quietTime[i],trkdata.rhythmpatterns[i]);
+   if (i != 10)  {printf("%d %d %d %d %d",trkdata.notepitchmin[i],  trkdata.notepitchmax[i] ,trkdata.notelengthmin[i],  trkdata.notelengthmax[i], trkdata.numberOfGaps[i]);
+   printf(" %f",trkdata.pitchEntropy[i]);
+    } else
+     printf("-1 0");
+   trkdata.quietTime[i] = 0; /* in case channel i is used in another track */
+   trkdata.numberOfGaps[i] = 0;
    printf("\n");
 
    channel2nnotes[i] += trkdata.notecount[i] + trkdata.chordcount[i];
@@ -595,19 +688,35 @@ void stats_trackstart()
   for (i=0;i<17;i++) {
      trkdata.notecount[i] = 0;
      trkdata.notemeanpitch[i] = 0;
+     trkdata.notepitchmin[i] = 128;
+     trkdata.notepitchmax[i] = 0;
      trkdata.notelength[i] = 0;
+     trkdata.notelengthmin[i] = 10000;
+     trkdata.notelengthmax[i] = 0;
      trkdata.chordcount[i] = 0;
      trkdata.cntlparam[i] = 0;
-     last_tick[i] = -1;
      last_on_tick[i] = -1;
+     channel_active[i] = 0;
      }
   printf("trk %d \n",tracknum);
+
+ for (i=0;i<2048;i++) lastTick[i] = -1;
+ for (i=0;i<17;i++) channel_used_in_track[i] = 0; /* [SS] 2023-09-06 */
+ for (i=0;i<204;i++) chanpitchhistogram[i] = 0;  /* [SS] 2023-09-13 */
 }
 
 void stats_trackend()
 {
- trkdata.npulses[tracknum] = Mf_currtime; 
+ int chan;
+ int i;
+ float entropy;
  if (trkdata.npulses[0] < Mf_currtime) trkdata.npulses[0] = Mf_currtime;
+ for (chan = 1; chan < 17; chan++) /* [SS] 2023-09-06 */
+   if (channel_used_in_track[chan] > 0) trkdata.quietTime[chan] += (trkdata.npulses[0] - trkdata.lastNoteOff[chan]); 
+ for (chan=0;chan<16;chan++) {  /* 2023-09-13 */
+   if (chan == 9 || channel_used_in_track[chan+1] == 0) continue;
+   trkdata.pitchEntropy[chan+1] = histogram_perplexity(chanpitchhistogram +chan*12,11);
+   }
  output_track_summary(); 
 }
 
@@ -619,25 +728,39 @@ int chan, pitch, vol;
  int delta;
  int barnum;
  int unit;
+ int dithermargin; /* [SS] 2023-08-22 */
+ int cpitch; /* [SS] 2023-09-13 */
+ int pulsePosition;
 
+ cpitch = pitch % 12;
+ channel_used_in_track[chan+1]++; /* [SS] 2023-09-06 */
+ dithermargin = unitDivision/2 - 1; 
  if (vol == 0) {
     /* treat as noteoff */
     stats_noteoff(chan,pitch,vol);
     trkdata.lastNoteOff[chan+1] = Mf_currtime; /* [SS] 2022.08.22 */
     return;
     }
+ pulsePosition = Mf_currtime % division;
+ pulseCounter[pulsePosition]++;
+ if (pulsePosition >= 1023) {printf("pulsePosition = %d too large\n",pulsePosition);
+     exit(1);
+     }
  trkdata.notemeanpitch[chan+1] += pitch;
+ trkdata.notepitchmax[chan+1] = max(trkdata.notepitchmax[chan+1],pitch);
+ trkdata.notepitchmin[chan+1] = min(trkdata.notepitchmin[chan+1],pitch);
  if (trkdata.lastNoteOff[chan+1] >= 0) {
 	 delta = Mf_currtime - trkdata.lastNoteOff[chan+1];
-	 trkdata.lastNoteOff[chan+1] = -1; /* in case of chord */
 	 if (delta > quietLimit) {
 		 trkdata.quietTime[chan+1] += delta;
+                 trkdata.numberOfGaps[chan+1]++;
+	         trkdata.lastNoteOff[chan+1] = -1; /* in case of chord */
 	 }
     }
 	 
  if (abs(Mf_currtime - last_on_tick[chan+1]) < chordthreshold) trkdata.chordcount[chan+1]++;
  else trkdata.notecount[chan+1]++; /* [SS] 2019-08-02 */
- last_tick[chan+1] = Mf_currtime;
+ lastTick[chan*128 + pitch] = Mf_currtime;
  last_on_tick[chan+1] = Mf_currtime; /* [SS] 2019-08-02 */
  /* last_on_tick not updated by stats_noteoff */
 
@@ -652,9 +775,10 @@ int chan, pitch, vol;
 	 barChn[chan].rhythmPattern = 0;
          barChn[chan].activeBarNumber = barnum;
     }
-  unit = (Mf_currtime % divisionsPerBar)/unitDivision;
+  unit = ((Mf_currtime+dithermargin) % divisionsPerBar)/unitDivision;
   //printf("unit = %d pattern = %d \n",unit,barChn[chan].rhythmPattern);
   barChn[chan].rhythmPattern = barChn[chan].rhythmPattern |= (1UL << unit);
+  chanpitchhistogram[chan*12+cpitch]++;  /* [SS] 2023-09-13 */
   }
 
 
@@ -665,6 +789,8 @@ int chan, pitch, vol;
    else  drumhistogram[pitch]++;
    }
  else pitchhistogram[pitch % 12]++;  /* [SS] 2017-11-01 */
+
+ channel_active[chan+1]++;
 }
 
 
@@ -677,17 +803,28 @@ void stats_noteoff(int chan,int pitch,int vol)
   int length;
   int program;
   /* ignore if there was no noteon */
-  if (last_tick[chan+1] == -1) return;
-  length = Mf_currtime - last_tick[chan+1];
+  if (lastTick[chan*128+pitch] == -1) return;
+  length = Mf_currtime - lastTick[chan*128+pitch];
   trkdata.notelength[chan+1] += length;
+  trkdata.notelengthmax[chan+1] = max(trkdata.notelengthmax[chan+1],length);
+  trkdata.notelengthmin[chan+1] = min(trkdata.notelengthmin[chan+1],length);
+  //if (length < 3) printf("chan = %d  lasttick = %d currtime = %ld\n",chan,lastTick[chan*128+pitch],Mf_currtime);
   trkdata.lastNoteOff[chan+1] = Mf_currtime; /* [SS] 2022.08.22 */
   chnactivity[chan+1] += length;
+  trkactivity[tracknum]++;
   if (chan == 9) return; /* drum channel */
   pitchclass_activity[pitch % 12] += length;
   program = trkdata.program[chan+1];
   progactivity[program] += length;
+  channel_active[chan+1]--;
   /* [SS] 2018-04-18 */
-  if(Mf_currtime > last_tick[chan+1]) last_tick[chan+1] = Mf_currtime;
+  if(Mf_currtime > lastTick[chan*128+pitch] && channel_active[chan+1] == 0) 
+    lastTick[chan*128+pitch] = Mf_currtime; /* [SS] 2023.08.30 handle chords */
+    
+  if (length > 4800) {
+     lastTick[chan*128+pitch] = Mf_currtime; /* handle stuck note [SS] 2023.08.30 */
+     channel_active[chan+1] = 0;
+     }
 }
 
 
@@ -721,7 +858,7 @@ if (trkdata.program[chan+1] != 0) {
   printf("program %d %d\n",chan+1,program);
   trkdata.program[chan+1] = program;
   }
-  if (channel2prog[chan+1]== -1) channel2prog[chan+1] = program;
+  if (channel2prog[chan+1]== 0) channel2prog[chan+1] = program; /* [SS] 2023-06-25*/
 }
 
 
@@ -729,11 +866,16 @@ if (trkdata.program[chan+1] != 0) {
 void stats_parameter(chan,control,value)
 int chan, control, value;
 {
-/*if (control == 7) {
-  printf("cntrlvolume %d %d \n",chan+1,value);
-  }
+int chan1;
+chan1 = chan+1;
+/* There may be many volume commands for the same channel. Only
+   record the first one.
 */
-trkdata.cntlparam[chan+1]++;
+if (control == 7 && trkdata.chanvol[chan1] == 0) {
+  /*printf("cntrlvolume %d %d \n",chan+1,value);*/
+  trkdata.chanvol[chan1] = value; /* [SS] 2023-10-30 */
+  }
+trkdata.cntlparam[chan1]++;
 }
 
 
@@ -743,6 +885,7 @@ int type, leng;
 char *mess;
 {
 int i; 
+if (type == 5) hasLyrics = 1; /* [SS] 2023-10-30 */
 if (type != 3) return;
 printf("metatext %d ",type);
 for (i=0;i<leng;i++) printf("%c",mess[i]);
@@ -805,9 +948,10 @@ midievents[lastEvent].channel = chan;
 midievents[lastEvent].pitch = pitch;
 midievents[lastEvent].velocity = vol;
 lastEvent++;
-if (lastEvent > 39999) {printf("ran out of space in midievents structure\n");
+if (lastEvent > 49999) {printf("ran out of space in midievents structure\n");
 	exit(1);
         }
+channel_active[chan+1]++;
 }
 
 void record_noteoff(int chan,int pitch,int vol)
@@ -816,6 +960,12 @@ void record_noteoff(int chan,int pitch,int vol)
 void record_trackend()
 {
 }
+void record_tempo(long ltempo)
+{ 
+tempo = ltempo;
+if (bpm == 120) bpm = 60000000.0/tempo;
+tempocount++;
+}
 
 int int_compare_events(const void *a, const void *b) {
    struct eventstruc *ia  = (struct eventstruc *)a;
@@ -829,12 +979,16 @@ int int_compare_events(const void *a, const void *b) {
 
 void load_header (int format, int ntrks, int ldivision)
 {
+  int i;
   division = ldivision;
+  lasttrack = ntrks;
+  for (i=0;i<17;i++) channel_active[i] = 0; /* for counting number of channels*/
 }
 
 
 void initfunc_for_stats()
 {
+    int i;
     Mf_error = stats_error; /* [SS] 2017-11-19 */
     Mf_header = stats_header;
     Mf_trackstart = stats_trackstart;
@@ -857,6 +1011,7 @@ void initfunc_for_stats()
     Mf_seqspecific = no_op3;
     Mf_text = stats_metatext;
     Mf_arbitrary = no_op2;
+    for (i = 0; i< 1023; i++) pulseCounter[i] = 0;
 }
 
 
@@ -879,7 +1034,7 @@ void initfunc_for_loadNoteEvents()
     Mf_eot = no_op0;
     Mf_timesig = no_op4;
     Mf_smpte = no_op5;
-    Mf_tempo = no_op1;
+    Mf_tempo = record_tempo;
     Mf_keysig = no_op2;
     Mf_seqspecific = no_op3;
     Mf_text = no_op3;
@@ -908,16 +1063,16 @@ int pulsePosition;
 int decimate;
 float fraction;
 int resolution = 12;
-for (i = 0; i< 480; i++) pulseCounter[i] = 0;
+for (i = 0; i< 1023; i++) pulseCounter[i] = 0;
 for (i = 0; i < lastEvent; i++) {
   pulsePosition = midievents[i].onsetTime % division;
   pulseCounter[pulsePosition]++;
-  if (pulsePosition >= 480) {printf("pulsePosition = %d too large\n",pulsePosition);
+  if (pulsePosition >= 1023) {printf("pulsePosition = %d too large\n",pulsePosition);
 	     exit(1);
              }
   }
 for (i = 0; i < resolution; i++) pulseDistribution[i] = 0;
-  /*for (i = 0; i < 480; i++) printf(" %d",pulseCounter[i]);
+  /*for (i = 0; i < 1023; i++) printf(" %d",pulseCounter[i]);
   printf("\n");
   */
 decimate = division/resolution;
@@ -953,7 +1108,6 @@ for (i = 0; i < lastEvent; i++) {
 }
 
 void drumpattern (int perc) {
-unsigned char drumpat[1000];
 int i;
 int channel;
 int pitch;
@@ -963,7 +1117,7 @@ int quarter;
 int remainder;
 int part;
 quarter = division/4;
-for (i = 0; i<1000; i++) drumpat[i] = 0;
+for (i = 0; i<8000; i++) drumpat[i] = 0;
 for (i = 0; i <lastEvent; i++) {
   channel = midievents[i].channel;
   if (channel != 9) continue;
@@ -973,12 +1127,92 @@ for (i = 0; i <lastEvent; i++) {
   index = onset/division;
   remainder = onset % division;
   part = remainder/quarter;
+  if (index >= 8000) {printf("index too large in drumpattern\n");
+	              break;
+                      }
+  part = 3 - part; /* order the bits going left to right */
   drumpat[index] = drumpat[index] |= 1 << part;
   }
+}
+
+void dualDrumPattern (int perc1, int perc2) {
+int i;
+int channel;
+int pitch;
+int onset;
+int index;
+int quarter;
+int remainder;
+int part;
+quarter = division/4;
+for (i = 0; i<8000; i++) drumpat[i] = 0;
+for (i = 0; i <lastEvent; i++) {
+  channel = midievents[i].channel;
+  if (channel != 9) continue;
+  pitch = midievents[i].pitch;
+  if (pitch != perc1  && pitch != perc2) continue;
+  onset = midievents[i].onsetTime;
+  index = onset/division;
+  if (index >= 8000) {printf("index too large in drumpattern\n");
+	              break;
+                      }
+  remainder = onset % division;
+  part = remainder/quarter;
+  part = 3 - part; /* order the bits from left to right */
+  if (pitch == perc1) drumpat[index] = drumpat[index] |= 1 << part;
+  else drumpat[index] = drumpat[index] |= 16 * (1 << part);
+  }
+}
+
+void pitchClassAnalysis () {
+int i;
+int channel;
+int pitch;
+for (i=0;i<12;i++) pitchclass_activity[i] = 0;
+for (i = 0; i < lastEvent; i++) {
+  channel = midievents[i].channel;
+  if (channel == 9) continue;
+  pitch = midievents[i].pitch;
+  pitchclass_activity[pitch % 12]++;
+  }
+}
+
+void output_perc_pattern (int i) {
+int left,right;
+left = i/16;
+right = i % 16;
+printf(" (%d.%d)",left,right);
+}
+
+
+void drumPatternHistogram () {
+int i;
+for (i=0;i<256;i++) {
+ histogram[i] = 0;
+}
+for (i=0;i<lastBeat;i++) {
+  histogram[drumpat[i]]++;
+  }
+for (i=1;i<256;i++) {
+  if (histogram[i] > 0) {
+	  printf(" %d",i);
+	  output_perc_pattern(i);
+	  printf(" %d ", histogram[i]);
+         }
+    }
+  printf("\n");
+}
+
+
+void output_drumpat () {
+int i;
 for (i=0;i<lastBeat;i++) printf("%d ",drumpat[i]);
+/*for (i=0;i<lastBeat;i++) output_perc_pattern(drumpat[i]);*/
 printf("\n");
 }
 
+
+
 void percsummary () {
 int i;
 int bassindex,snareindex;
@@ -1002,13 +1236,31 @@ if (bassmax && snaremax) {
 	printf("snare %d %d\n",snareindex,snaremax);
    } else {
         printf("missing bass or snare\n");
+	return;
    }
-			
+dualDrumPattern(bassindex,snareindex);
 }
 
+void outputPitchClassHistogram() {
+int i;
+float activity;
+activity = 0;
+for (i=0;i<12;i++) activity += (float) pitchclass_activity[i];
+for (i=0;i<11;i++) printf("%5.3f,",pitchclass_activity[i]/activity);
+printf("%5.3f",pitchclass_activity[11]/activity);
+printf("\n");
+}
+
+
   
 void corestatsOutput() {
-printf("%d\t%d\t%d\n", division,lastEvent,lastBeat);
+int i;
+int nchannels;
+nchannels = 0;
+for (i=1;i<17;i++) 
+  if (channel_active[i] > 0) nchannels++;
+printf("%d\t%d\t%d\t%d\t%d\t%d\n",lasttrack,nchannels, division,bpm,lastEvent,lastBeat);
+/*printf("%d\n",tempocount);*/
 }
 
 
@@ -1129,18 +1381,41 @@ int argc;
 	  stats = 0;
           }
 
-  arg = getarg("-percanalysis",argc,argv);
+  arg = getarg("-panal",argc,argv);
   if (arg != -1) {
 	  percanalysis = 1;
 	  stats = 0;
           }
 
-  arg = getarg("-percpattern",argc,argv);
+  arg = getarg("-ppat",argc,argv);
   if (arg != -1) {
 	  percpattern = 1;
 	  stats = 0;
           }
 
+  arg = getarg("-ppatfor",argc,argv);
+  if (arg != -1) {
+	  percpatternfor = 1;
+	  stats = 0;
+	  if (arg != -1 && arg <argc) {
+		  percnum = readnum(argv[arg]);
+	          printf("percnum = %d\n",percnum);
+                  }
+          }
+
+  arg = getarg("-ppathist",argc,argv);
+  if (arg != -1) {
+	  percpatternhist = 1;
+	  stats = 0;
+          }
+
+  arg = getarg("-pitchclass",argc,argv);
+  if (arg != -1) {
+	  pitchclassanalysis = 1;
+	  stats = 0;
+          }
+
+
   arg = getarg("-o",argc,argv);
   if ((arg != -1) && (arg < argc))  {
     outhandle = efopen(argv[arg],"w");  /* open output abc file */
@@ -1163,12 +1438,15 @@ int argc;
     printf("midistats filename <options>\n");
     printf("         -corestats\n");
     printf("         -pulseanalysis\n");
-    printf("         -percanalysis\n");
-    printf("         -percpattern\n");
+    printf("         -panal\n");
+    printf("         -ppat\n");
+    printf("         -ppatfor\n");
+    printf("         -ppathist\n");
+    printf("         -pitchclass\n");
     printf("         -ver version number\n");
     printf("         -d <number> debug parameter\n");
     printf(" The input filename is assumed to be any string not\n");
-    printf(" beginning with a - (hyphen). It may be placed anywhere.\n");
+    printf(" beginning with a - (hyphen).\n");
     exit(0);
   };
   return arg;
@@ -1202,12 +1480,25 @@ if (percanalysis) {
         }
         printf("\n");
       }
-if (percanalysis) drumpattern(40);
 if (percpattern) {
     drumanalysis();
     percsummary();
+    output_drumpat();
+    }
+if (percpatternfor) {
+    drumpattern(percnum);
+    output_drumpat();
+    }
+if (percpatternhist) {
+    drumanalysis();
+    percsummary();
+    drumPatternHistogram();
     }
 if (corestats) corestatsOutput();
+if (pitchclassanalysis) {
+   pitchClassAnalysis();
+   outputPitchClassHistogram(); 
+   }
 }
 
 
@@ -1221,6 +1512,8 @@ int argc;
 
   arg = process_command_line_arguments(argc,argv);
   if(stats == 1)  midistats(argc,argv);
-  if(pulseanalysis || corestats || percanalysis || percpattern) loadEvents();
+  if(pulseanalysis || corestats || percanalysis ||\
+    percpatternfor || percpattern || percpatternhist ||\
+    pitchclassanalysis) loadEvents();
   return 0;
 }