promo.morph_utils.morph_sequence API documentation

# promo.morph_utils.morph_sequence

Created on Sep 18, 2013

@author: timmahrt

Given two lists of tuples of the form [(value, time), (value, time)], morph can iteratively transform the values in one list to the values in the other while maintaining the times in the first list.

Both time scales are placed on a relative scale. This assumes that the times may be different and the number of samples may be different but the 'events' occur at the same relative location (half way through, at the end, etc.).

Both dynamic time warping and morph, align two data lists in time. However, dynamic time warping does this by analyzing the event structure and aligning events in the two signals as best it can (i.e. it changes when events happen in relative time while morph preserves when events happen in relative time).

View Source

      1"""
      2Created on Sep 18, 2013
      3
      4@author: timmahrt
      5
      6Given two lists of tuples of the form [(value, time), (value, time)], morph
      7can iteratively transform the values in one list to the values in the other
      8while maintaining the times in the first list.
      9
     10Both time scales are placed on a relative scale.  This assumes that the times
     11may be different and the number of samples may be different but the 'events'
     12occur at the same relative location (half way through, at the end, etc.).
     13
     14Both dynamic time warping and morph, align two data lists in time.  However,
     15dynamic time warping does this by analyzing the event structure and aligning
     16events in the two signals as best it can
     17(i.e. it changes when events happen in relative time while morph preserves
     18when events happen in relative time).
     19"""
     20
     21
     22class RelativizeSequenceException(Exception):
     23    def __init__(self, dist):
     24        super(RelativizeSequenceException, self).__init__()
     25        self.dist = dist
     26
     27    def __str__(self):
     28        return (
     29            "You need at least two unique values to make "
     30            + "a sequence relative. Input: %s" % repr(self.dist)
     31        )
     32
     33
     34def makeSequenceRelative(absVSequence):
     35    """
     36    Puts every value in a list on a continuum between 0 and 1
     37
     38    Also returns the min and max values (to reverse the process)
     39    """
     40
     41    if len(absVSequence) < 2 or len(set(absVSequence)) == 1:
     42        raise RelativizeSequenceException(absVSequence)
     43
     44    minV = min(absVSequence)
     45    maxV = max(absVSequence)
     46    relativeSeq = [(value - minV) / (maxV - minV) for value in absVSequence]
     47
     48    return relativeSeq, minV, maxV
     49
     50
     51def makeSequenceAbsolute(relVSequence, minV, maxV):
     52    """
     53    Makes every value in a sequence absolute
     54    """
     55
     56    return [(value * (maxV - minV)) + minV for value in relVSequence]
     57
     58
     59def _makeTimingRelative(absoluteDataList):
     60    """
     61    Given normal pitch tier data, puts the times on a scale from 0 to 1
     62
     63    Input is a list of tuples of the form
     64    ([(time1, pitch1), (time2, pitch2),...]
     65
     66    Also returns the start and end time so that the process can be reversed
     67    """
     68
     69    timingSeq = [row[0] for row in absoluteDataList]
     70    valueSeq = [list(row[1:]) for row in absoluteDataList]
     71
     72    relTimingSeq, startTime, endTime = makeSequenceRelative(timingSeq)
     73
     74    relDataList = [
     75        tuple(
     76            [
     77                time,
     78            ]
     79            + row
     80        )
     81        for time, row in zip(relTimingSeq, valueSeq)
     82    ]
     83
     84    return relDataList, startTime, endTime
     85
     86
     87def _makeTimingAbsolute(relativeDataList, startTime, endTime):
     88    """
     89    Maps values from 0 to 1 to the provided start and end time
     90
     91    Input is a list of tuples of the form
     92    ([(time1, pitch1), (time2, pitch2),...]
     93    """
     94
     95    timingSeq = [row[0] for row in relativeDataList]
     96    valueSeq = [list(row[1:]) for row in relativeDataList]
     97
     98    absTimingSeq = makeSequenceAbsolute(timingSeq, startTime, endTime)
     99
    100    absDataList = [
    101        tuple(
    102            [
    103                time,
    104            ]
    105            + row
    106        )
    107        for time, row in zip(absTimingSeq, valueSeq)
    108    ]
    109
    110    return absDataList
    111
    112
    113def _getSmallestDifference(inputList, targetVal):
    114    """
    115    Returns the value in inputList that is closest to targetVal
    116
    117    Iteratively splits the dataset in two, so it should be pretty fast
    118    """
    119    targetList = inputList[:]
    120    retVal = None
    121    while True:
    122        # If we're down to one value, stop iterating
    123        if len(targetList) == 1:
    124            retVal = targetList[0]
    125            break
    126        halfPoint = int(len(targetList) / 2.0) - 1
    127        a = targetList[halfPoint]
    128        b = targetList[halfPoint + 1]
    129
    130        leftDiff = abs(targetVal - a)
    131        rightDiff = abs(targetVal - b)
    132
    133        # If the distance is 0, stop iterating, the targetVal is present
    134        # in the inputList
    135        if leftDiff == 0 or rightDiff == 0:
    136            retVal = targetVal
    137            break
    138
    139        # Look at left half or right half
    140        if leftDiff < rightDiff:
    141            targetList = targetList[: halfPoint + 1]
    142        else:
    143            targetList = targetList[halfPoint + 1 :]
    144
    145    return retVal
    146
    147
    148def _getNearestMappingIndexList(fromValList, toValList):
    149    """
    150    Finds the indicies for data points that are closest to each other.
    151
    152    The inputs should be in relative time, scaled from 0 to 1
    153    e.g. if you have [0, .1, .5., .9] and [0, .1, .2, 1]
    154    will output [0, 1, 1, 2]
    155    """
    156
    157    indexList = []
    158    for fromTimestamp in fromValList:
    159        smallestDiff = _getSmallestDifference(toValList, fromTimestamp)
    160        i = toValList.index(smallestDiff)
    161        indexList.append(i)
    162
    163    return indexList
    164
    165
    166def morphDataLists(fromList, toList, stepList):
    167    """
    168    Iteratively morph fromList into toList using the values 0 to 1 in stepList
    169
    170    stepList: a value of 0 means no change and a value of 1 means a complete
    171    change to the other value
    172    """
    173
    174    # If there are more than 1 pitch value, then we align the data in
    175    # relative time.
    176    # Each data point comes with a timestamp.  The earliest timestamp is 0
    177    # and the latest timestamp is 1.  Using this method, for each relative
    178    # timestamp in the source list, we find the closest relative timestamp
    179    # in the target list.  Just because two pitch values have the same index
    180    # in the source and target lists does not mean that they correspond to
    181    # the same speech event.
    182    fromListRel, fromStartTime, fromEndTime = _makeTimingRelative(fromList)
    183    toListRel = _makeTimingRelative(toList)[0]
    184
    185    # If fromList has more points, we'll have flat areas
    186    # If toList has more points, we'll might miss peaks or valleys
    187    fromTimeList = [dataTuple[0] for dataTuple in fromListRel]
    188    toTimeList = [dataTuple[0] for dataTuple in toListRel]
    189    indexList = _getNearestMappingIndexList(fromTimeList, toTimeList)
    190    alignedToPitchRel = [toListRel[i] for i in indexList]
    191
    192    for stepAmount in stepList:
    193        newPitchList = []
    194
    195        # Perform the interpolation
    196        for fromTuple, toTuple in zip(fromListRel, alignedToPitchRel):
    197            fromTime, fromValue = fromTuple
    198            toTime, toValue = toTuple
    199
    200            # i + 1 b/c i_0 = 0 = no change
    201            newValue = fromValue + (stepAmount * (toValue - fromValue))
    202            newTime = fromTime + (stepAmount * (toTime - fromTime))
    203
    204            newPitchList.append((newTime, newValue))
    205
    206        newPitchList = _makeTimingAbsolute(newPitchList, fromStartTime, fromEndTime)
    207
    208        yield stepAmount, newPitchList
    209
    210
    211def morphChunkedDataLists(fromDataList, toDataList, stepList):
    212    """
    213    Morph one set of data into another, in a stepwise fashion
    214
    215    A convenience function.  Given a set of paired data lists,
    216    this will morph each one individually.
    217
    218    Returns a single list with all data combined together.
    219    """
    220
    221    assert len(fromDataList) == len(toDataList)
    222
    223    # Morph the fromDataList into the toDataList
    224    outputList = []
    225    for x, y in zip(fromDataList, toDataList):
    226
    227        # We cannot morph a region if there is no data or only
    228        # a single data point for either side
    229        if (len(x) < 2) or (len(y) < 2):
    230            continue
    231
    232        tmpList = [
    233            outputPitchList for _, outputPitchList in morphDataLists(x, y, stepList)
    234        ]
    235        outputList.append(tmpList)
    236
    237    # Transpose list
    238    finalOutputList = outputList.pop(0)
    239    for subList in outputList:
    240        for i, subsubList in enumerate(subList):
    241            finalOutputList[i].extend(subsubList)
    242
    243    return finalOutputList
    244
    245
    246def morphAveragePitch(fromDataList, toDataList):
    247    """
    248    Adjusts the values in fromPitchList to have the same average as toPitchList
    249
    250    Because other manipulations can alter the average pitch, morphing the pitch
    251    is the last pitch manipulation that should be done
    252
    253    After the morphing, the code removes any values below zero, thus the
    254    final average might not match the target average.
    255    """
    256
    257    timeList, fromPitchList = zip(*fromDataList)
    258    toPitchList = [pitchVal for _, pitchVal in toDataList]
    259
    260    # Zero pitch values aren't meaningful, so filter them out if they are
    261    # in the dataset
    262    fromListNoZeroes = [val for val in fromPitchList if val > 0]
    263    fromAverage = sum(fromListNoZeroes) / float(len(fromListNoZeroes))
    264
    265    toListNoZeroes = [val for val in toPitchList if val > 0]
    266    toAverage = sum(toListNoZeroes) / float(len(toListNoZeroes))
    267
    268    newPitchList = [val - fromAverage + toAverage for val in fromPitchList]
    269
    270    #     finalAverage = sum(newPitchList) / float(len(newPitchList))
    271
    272    # Removing zeroes and negative pitch values
    273    retDataList = [
    274        (time, pitchVal)
    275        for time, pitchVal in zip(timeList, newPitchList)
    276        if pitchVal > 0
    277    ]
    278
    279    return retDataList
    280
    281
    282def morphRange(fromDataList, toDataList):
    283    """
    284    Changes the scale of values in one distribution to that of another
    285
    286    ie The maximum value in fromDataList will be set to the maximum value in
    287    toDataList.  The 75% largest value in fromDataList will be set to the
    288    75% largest value in toDataList, etc.
    289
    290    Small sample sizes will yield results that are not very meaningful
    291    """
    292
    293    # Isolate and sort pitch values
    294    fromPitchList = [dataTuple[1] for dataTuple in fromDataList]
    295    toPitchList = [dataTuple[1] for dataTuple in toDataList]
    296
    297    fromPitchListSorted = sorted(fromPitchList)
    298    toPitchListSorted = sorted(toPitchList)
    299
    300    # Bin pitch values between 0 and 1
    301    fromListRel = makeSequenceRelative(fromPitchListSorted)[0]
    302    toListRel = makeSequenceRelative(toPitchListSorted)[0]
    303
    304    # Find each values closest equivalent in the other list
    305    indexList = _getNearestMappingIndexList(fromListRel, toListRel)
    306
    307    # Map the source pitch to the target pitch value
    308    # Pitch value -> get sorted position -> get corresponding position in
    309    # target list -> get corresponding pitch value = the new pitch value
    310    retList = []
    311    for time, pitch in fromDataList:
    312        fromI = fromPitchListSorted.index(pitch)
    313        toI = indexList[fromI]
    314        newPitch = toPitchListSorted[toI]
    315
    316        retList.append((time, newPitch))
    317
    318    return retList

class RelativizeSequenceException(builtins.Exception): View Source

    23class RelativizeSequenceException(Exception):
    24    def __init__(self, dist):
    25        super(RelativizeSequenceException, self).__init__()
    26        self.dist = dist
    27
    28    def __str__(self):
    29        return (
    30            "You need at least two unique values to make "
    31            + "a sequence relative. Input: %s" % repr(self.dist)
    32        )

Common base class for all non-exit exceptions.

RelativizeSequenceException(dist) View Source

    24    def __init__(self, dist):
    25        super(RelativizeSequenceException, self).__init__()
    26        self.dist = dist

dist

##### Inherited Members

builtins.BaseException
  with_traceback
  add_note
  args

def makeSequenceRelative(absVSequence): View Source

    35def makeSequenceRelative(absVSequence):
    36    """
    37    Puts every value in a list on a continuum between 0 and 1
    38
    39    Also returns the min and max values (to reverse the process)
    40    """
    41
    42    if len(absVSequence) < 2 or len(set(absVSequence)) == 1:
    43        raise RelativizeSequenceException(absVSequence)
    44
    45    minV = min(absVSequence)
    46    maxV = max(absVSequence)
    47    relativeSeq = [(value - minV) / (maxV - minV) for value in absVSequence]
    48
    49    return relativeSeq, minV, maxV

Puts every value in a list on a continuum between 0 and 1

Also returns the min and max values (to reverse the process)

def makeSequenceAbsolute(relVSequence, minV, maxV): View Source

    52def makeSequenceAbsolute(relVSequence, minV, maxV):
    53    """
    54    Makes every value in a sequence absolute
    55    """
    56
    57    return [(value * (maxV - minV)) + minV for value in relVSequence]

Makes every value in a sequence absolute

def morphDataLists(fromList, toList, stepList): View Source

    167def morphDataLists(fromList, toList, stepList):
    168    """
    169    Iteratively morph fromList into toList using the values 0 to 1 in stepList
    170
    171    stepList: a value of 0 means no change and a value of 1 means a complete
    172    change to the other value
    173    """
    174
    175    # If there are more than 1 pitch value, then we align the data in
    176    # relative time.
    177    # Each data point comes with a timestamp.  The earliest timestamp is 0
    178    # and the latest timestamp is 1.  Using this method, for each relative
    179    # timestamp in the source list, we find the closest relative timestamp
    180    # in the target list.  Just because two pitch values have the same index
    181    # in the source and target lists does not mean that they correspond to
    182    # the same speech event.
    183    fromListRel, fromStartTime, fromEndTime = _makeTimingRelative(fromList)
    184    toListRel = _makeTimingRelative(toList)[0]
    185
    186    # If fromList has more points, we'll have flat areas
    187    # If toList has more points, we'll might miss peaks or valleys
    188    fromTimeList = [dataTuple[0] for dataTuple in fromListRel]
    189    toTimeList = [dataTuple[0] for dataTuple in toListRel]
    190    indexList = _getNearestMappingIndexList(fromTimeList, toTimeList)
    191    alignedToPitchRel = [toListRel[i] for i in indexList]
    192
    193    for stepAmount in stepList:
    194        newPitchList = []
    195
    196        # Perform the interpolation
    197        for fromTuple, toTuple in zip(fromListRel, alignedToPitchRel):
    198            fromTime, fromValue = fromTuple
    199            toTime, toValue = toTuple
    200
    201            # i + 1 b/c i_0 = 0 = no change
    202            newValue = fromValue + (stepAmount * (toValue - fromValue))
    203            newTime = fromTime + (stepAmount * (toTime - fromTime))
    204
    205            newPitchList.append((newTime, newValue))
    206
    207        newPitchList = _makeTimingAbsolute(newPitchList, fromStartTime, fromEndTime)
    208
    209        yield stepAmount, newPitchList

Iteratively morph fromList into toList using the values 0 to 1 in stepList

stepList: a value of 0 means no change and a value of 1 means a complete change to the other value

def morphChunkedDataLists(fromDataList, toDataList, stepList): View Source

    212def morphChunkedDataLists(fromDataList, toDataList, stepList):
    213    """
    214    Morph one set of data into another, in a stepwise fashion
    215
    216    A convenience function.  Given a set of paired data lists,
    217    this will morph each one individually.
    218
    219    Returns a single list with all data combined together.
    220    """
    221
    222    assert len(fromDataList) == len(toDataList)
    223
    224    # Morph the fromDataList into the toDataList
    225    outputList = []
    226    for x, y in zip(fromDataList, toDataList):
    227
    228        # We cannot morph a region if there is no data or only
    229        # a single data point for either side
    230        if (len(x) < 2) or (len(y) < 2):
    231            continue
    232
    233        tmpList = [
    234            outputPitchList for _, outputPitchList in morphDataLists(x, y, stepList)
    235        ]
    236        outputList.append(tmpList)
    237
    238    # Transpose list
    239    finalOutputList = outputList.pop(0)
    240    for subList in outputList:
    241        for i, subsubList in enumerate(subList):
    242            finalOutputList[i].extend(subsubList)
    243
    244    return finalOutputList

Morph one set of data into another, in a stepwise fashion

A convenience function. Given a set of paired data lists, this will morph each one individually.

Returns a single list with all data combined together.

def morphAveragePitch(fromDataList, toDataList): View Source

    247def morphAveragePitch(fromDataList, toDataList):
    248    """
    249    Adjusts the values in fromPitchList to have the same average as toPitchList
    250
    251    Because other manipulations can alter the average pitch, morphing the pitch
    252    is the last pitch manipulation that should be done
    253
    254    After the morphing, the code removes any values below zero, thus the
    255    final average might not match the target average.
    256    """
    257
    258    timeList, fromPitchList = zip(*fromDataList)
    259    toPitchList = [pitchVal for _, pitchVal in toDataList]
    260
    261    # Zero pitch values aren't meaningful, so filter them out if they are
    262    # in the dataset
    263    fromListNoZeroes = [val for val in fromPitchList if val > 0]
    264    fromAverage = sum(fromListNoZeroes) / float(len(fromListNoZeroes))
    265
    266    toListNoZeroes = [val for val in toPitchList if val > 0]
    267    toAverage = sum(toListNoZeroes) / float(len(toListNoZeroes))
    268
    269    newPitchList = [val - fromAverage + toAverage for val in fromPitchList]
    270
    271    #     finalAverage = sum(newPitchList) / float(len(newPitchList))
    272
    273    # Removing zeroes and negative pitch values
    274    retDataList = [
    275        (time, pitchVal)
    276        for time, pitchVal in zip(timeList, newPitchList)
    277        if pitchVal > 0
    278    ]
    279
    280    return retDataList

Adjusts the values in fromPitchList to have the same average as toPitchList

Because other manipulations can alter the average pitch, morphing the pitch is the last pitch manipulation that should be done

After the morphing, the code removes any values below zero, thus the final average might not match the target average.

def morphRange(fromDataList, toDataList): View Source

    283def morphRange(fromDataList, toDataList):
    284    """
    285    Changes the scale of values in one distribution to that of another
    286
    287    ie The maximum value in fromDataList will be set to the maximum value in
    288    toDataList.  The 75% largest value in fromDataList will be set to the
    289    75% largest value in toDataList, etc.
    290
    291    Small sample sizes will yield results that are not very meaningful
    292    """
    293
    294    # Isolate and sort pitch values
    295    fromPitchList = [dataTuple[1] for dataTuple in fromDataList]
    296    toPitchList = [dataTuple[1] for dataTuple in toDataList]
    297
    298    fromPitchListSorted = sorted(fromPitchList)
    299    toPitchListSorted = sorted(toPitchList)
    300
    301    # Bin pitch values between 0 and 1
    302    fromListRel = makeSequenceRelative(fromPitchListSorted)[0]
    303    toListRel = makeSequenceRelative(toPitchListSorted)[0]
    304
    305    # Find each values closest equivalent in the other list
    306    indexList = _getNearestMappingIndexList(fromListRel, toListRel)
    307
    308    # Map the source pitch to the target pitch value
    309    # Pitch value -> get sorted position -> get corresponding position in
    310    # target list -> get corresponding pitch value = the new pitch value
    311    retList = []
    312    for time, pitch in fromDataList:
    313        fromI = fromPitchListSorted.index(pitch)
    314        toI = indexList[fromI]
    315        newPitch = toPitchListSorted[toI]
    316
    317        retList.append((time, newPitch))
    318
    319    return retList

Changes the scale of values in one distribution to that of another

ie The maximum value in fromDataList will be set to the maximum value in toDataList. The 75% largest value in fromDataList will be set to the 75% largest value in toDataList, etc.

Small sample sizes will yield results that are not very meaningful