Handle Spike or overflow value in dataset

Is the rollover point a system parameter that you could get? How is that normally determined?
Sudden drop will occur. i have also given example

What are you wanting on maxAllowed increment?
i don't know - max allowed we need to use


like this we need to calculate through scirpting

I think this should be pushed back to the PLC. It needs to provide an odometer value that doesn't change during meter restart. "Cleaning up" this signal in Ignition is unlikely to be reliable.

5 Likes

Its Totalizer tag. For meter consumption Report based on the time frame. I need this feature

You are getting garbage data in. You are very likely to be unable to prevent garbage data out. Ignition is not a magic wand.

2 Likes

Yes i understood. But thought of handle it through scripting for reporting part

Consider using an expression tag that will force bad quality when a new value is impossibly different from the last good value. Historize the expression tag instead of the original.

1 Like

What triggers the rollover? Is it a button push? Is it a timer? Is it shift change? Is it random?

Where is the logic for the control that does the reset? Is it an HMI interface? Is it a PLC? Is it Ignition?

How much trouble are you going to get into if the algorithm malfunctions and logs an errant report? Will you fail an audit? Will you lose customers? Will you overcharge somebody?

Even if the report is not a critical control point, it's probably worth taking a step back to examine your implementation to see if there is a more reliable method. This code requirement looks unsafe.

2 Likes

Ok will check that

Understood will check with my team

1 Like

This is as dynamic as I can get it. It calculates the maximum acceptable increment by making a list of all of the differences, and taking the mean of the middle 50 percent of the values. Then it makes a list of all the indexes where a legitimate rollover takes place, and creates a list of sums for each set of numbers from rollover to rollover, just like you are currently doing on your datasheet. Finally, it sums the list to produce a total.

Here is the code: (Use it at your own risk)

from java.util import Collections
def getMaxAllowedIncrement(data):
	listOfDifferences = [float(data[index] - data[index-1]) for index in range(1, len(data)) if data[index] - data[index-1] >= 0]
	Collections.sort(listOfDifferences)
	filteredDifferences = [
		listOfDifferences[index]
		for index in range(int(.25 * float(len(listOfDifferences))), int(.75 * float(len(listOfDifferences))))
	]
	averageFilteredDifference = sum(filteredDifferences)/len(filteredDifferences)
	plusOrMinusDifference = 2 * averageFilteredDifference
	return plusOrMinusDifference
def getRolloverPoints(data, maxAllowedIncrement):
	rolloverPoints = []
	for index in range(1, len(data)):
		change = data[index] - data[index-1]
		if change < 0 and index-2 >=0 and index + 1 < len(data):
			if abs(data[index-1] - data[index-2]) < maxAllowedIncrement and abs(data[index] - data[index + 1]) < maxAllowedIncrement:
				rolloverPoints.append(index-1)
	return rolloverPoints
data = dataset.getColumnAsList(dataset.getColumnIndex('value'))
if len(data) > 1:
	maxAllowedIncrement = getMaxAllowedIncrement(data)
	rolloverPoints = getRolloverPoints(data, maxAllowedIncrement)
	startingValue = data[0]
	finalValue = data[len(data)-1]
	listOfTotals = []
	if len(rolloverPoints) > 0:
		firstRollover = data[rolloverPoints[0]]
		listOfTotals.append(firstRollover - startingValue)
		for index in range(2, len(rolloverPoints)):
			previousRolloverIndex = rolloverPoints[index-1]
			startinRolloverValue = data[previousRolloverIndex+1]
			nextRolloverValue = data[rolloverPoints[index]]
			listOfTotals.append(nextRolloverValue - startinRolloverValue)
		if rolloverPoints[len(rolloverPoints)-1]+1 < len(data)-1:
			
			listOfTotals.append(data[len(data)-1] - data[rolloverPoints[len(rolloverPoints)-1]+1])
	else:
		listOfTotals.append(finalValue - startingValue)
	totalKWH = sum(listOfTotals)
	print listOfTotals
	print totalKWH

Output:
[1575, 2630]
4205

From your spreadsheet:
image

image

image

Performing the calculation manually produces an exact match to the script output:
9999999 - 9998424 = 1575
2630 - 0 = 2630
1575 + 2630 = 4205

1 Like

This is the code i have tired

def find_incremental_sequences(lst):
    # Initializations
    sequences = []
    current_sequence = []

    # Iterate over the list
    for i in range(len(lst)):
        if i == 0 or lst[i] >= lst[i-1]:
            current_sequence.append(lst[i])
        else:
            if len(current_sequence) > 1:
                sequences.append((current_sequence[0], current_sequence[-1]))
            current_sequence = [lst[i]]

    if len(current_sequence) > 1:
        sequences.append((current_sequence[0], current_sequence[-1]))

    return sequences

# Example usage
my_list = [3, 5, 8, 12, 15, 10, 0, 2, 3, 4, 5, 6, 0, 20, 30, 70]
incremental_sequences = find_incremental_sequences(my_list)

# Calculate the differences and sum them
difference_sum = sum(last_val - first_val for first_val, last_val in incremental_sequences)

# Print the first and last value of each incremental sequence
for i, (first_val, last_val) in enumerate(incremental_sequences):
    print("Incremental Sequence {}: First = {}, Last = {}".format(i+1, first_val, last_val))

print("Sum of differences between first and last values:", difference_sum)

It's also giving correct output

To retrieve the first and last value of each incremental sequence list and it does the calculation

I tested it with the test data list from your spreadsheet:

my_list = [9998424, 9998425, 9998426, 9998427, 9998428, 9998429, 9998430, 9998431, 9998432, 9998433, 9998434, 9998435, 9998436, 9998437, 9998438, 9998439, 9998440, 9998441, 9998442, 9998444, 9998445, 9998446, 9998447, 9998448, 9998449, 9998450, 9998451, 9998452, 9998453, 9998454, 9998455, 9998456, 9998457, 9998458, 9998459, 9998460, 9998461, 9998462, 9998463, 9998464, 9998465, 9998466, 9998467, 9998468, 9998469, 9998470, 9998471, 9998472, 9998473, 9998474, 9998475, 9998476, 9998477, 9998478, 9998479, 9998480, 9998481, 9998482, 9998483, 9998484, 9998486, 9998487, 9998488, 9998489, 9998490, 9998491, 9998492, 9998493, 9998494, 9998495, 9998496, 9998497, 9998498, 9998499, 9998500, 9998501, 9998502, 9998503, 9998504, 9998505, 9998506, 9998507, 9998508, 9998509, 9998510, 9998511, 9998512, 9998513, 9998514, 9998515, 9998516, 9998517, 9998518, 9998519, 9998520, 9998521, 9998522, 9998523, 9998524, 9998525, 9998526, 9998527, 9998529, 9998530, 9998531, 9998532, 9998533, 9998534, 9998535, 9998536, 9998537, 9998538, 9998539, 9998540, 9998541, 9998542, 9998543, 9998544, 9998545, 9998546, 9998547, 9998548, 9998549, 9998550, 9998551, 9998552, 9998553, 9998554, 9998555, 9998556, 9998557, 9998558, 9998559, 9998560, 9998561, 9998562, 9998563, 9998564, 9998565, 9998566, 9998567, 9998568, 9998569, 9998570, 9998572, 9998573, 9998574, 9998575, 9998576, 9998577, 9998578, 9998579, 9998580, 9998581, 9998582, 9998583, 9998584, 9998585, 9998586, 9998587, 9998588, 9998589, 9998590, 9998591, 9998592, 9998593, 9998594, 9998595, 9998596, 9998597, 9998598, 9998599, 9998600, 9998601, 9998602, 9998603, 9998604, 9998605, 9998606, 9998607, 9998608, 9998609, 9998610, 9998611, 9998612, 9998614, 9998615, 9998616, 9998617, 9998618, 9998619, 9998620, 9998621, 9998622, 9998623, 9998624, 9998625, 9998626, 9998627, 9998628, 9998629, 9998630, 9998631, 9998632, 9998633, 9998634, 9998635, 9998636, 9998637, 9998638, 9998639, 9998640, 9998641, 9998642, 9998643, 9998644, 9998645, 9998646, 9998647, 9998648, 9998649, 9998650, 9998651, 9998652, 9998653, 9998654, 9998655, 9998657, 9998658, 9998659, 9998660, 9998661, 9998662, 9998663, 9998664, 9998665, 9998666, 9998667, 9998668, 9998669, 9998670, 9998671, 9998672, 9998673, 9998674, 9998675, 9998676, 9998677, 9998678, 9998679, 9998680, 9998681, 9998682, 9998683, 9998684, 9998685, 9998686, 9998687, 9998688, 9998689, 9998690, 9998691, 9998692, 9998693, 9998694, 9998695, 9998696, 9998697, 9998698, 9998700, 9998701, 9998702, 9998703, 9998704, 9998705, 9998706, 9998707, 9998708, 9998709, 9998710, 9998711, 9998712, 9998713, 9998714, 9998715, 9998716, 9998717, 9998718, 9998719, 9998720, 9998721, 9998722, 9998723, 9998724, 9998725, 9998726, 9998727, 9998728, 9998729, 9998730, 9998731, 9998732, 9998733, 9998734, 9998735, 9998736, 9998737, 9998738, 9998739, 9998740, 9998742, 9998743, 9998744, 9998745, 9998746, 9998747, 9998748, 9998749, 9998750, 9998751, 9998752, 9998753, 9998754, 9998755, 9998756, 9998757, 9998758, 9998759, 9998760, 9998761, 9998762, 9998763, 9998764, 9998765, 9998766, 9998767, 9998768, 9998769, 9998770, 9998771, 9998772, 9998773, 9998774, 9998775, 9998776, 9998777, 9998778, 9998779, 9998780, 9998781, 9998782, 9998783, 9998785, 9998786, 9998787, 9998788, 9998789, 9998790, 9998791, 9998792, 9998793, 9998794, 9998795, 9998796, 9998797, 9998798, 9998799, 9998800, 9998801, 9998802, 9998803, 9998804, 9998805, 9998806, 9998807, 9998808, 9998809, 9998810, 9998811, 9998812, 9998813, 9998814, 9998815, 9998816, 9998817, 9998818, 9998819, 9998820, 9998821, 9998822, 9998823, 9998824, 9998825, 9998826, 9998828, 9998829, 9998830, 9998831, 9998832, 9998833, 9998834, 9998835, 9998836, 9998837, 9998838, 9998839, 9998840, 9998841, 9998842, 9998843, 9998844, 9998845, 9998846, 9998847, 9998848, 9998849, 9998850, 9998851, 9998852, 9998853, 9998854, 9998855, 9998856, 9998857, 9998858, 9998859, 9998860, 9998861, 9998862, 9998863, 9998864, 9998865, 9998866, 9998867, 9998868, 9998870, 9998871, 9998872, 9998873, 9998874, 9998875, 9998876, 9998877, 9998878, 9998879, 9998880, 9998881, 9998882, 9998883, 9998884, 9998885, 9998886, 9998887, 9998888, 9998889, 9998890, 9998891, 9998892, 9998893, 9998894, 9998895, 9998896, 9998897, 9998898, 9998899, 9998900, 9998901, 9998902, 9998903, 9998904, 9998905, 9998906, 9998907, 9998908, 9998909, 9998910, 9998911, 9998913, 9998914, 9998915, 9998916, 9998917, 9998918, 9998919, 9998920, 9998921, 9998922, 9998923, 9998924, 9998925, 9998926, 9998927, 9998928, 9998929, 9998930, 9998931, 9998932, 9998933, 9998934, 9998935, 9998936, 9998937, 9998938, 9998939, 9998940, 9998941, 9998942, 9998943, 9998944, 9998945, 9998946, 9998947, 9998948, 9998949, 9998950, 9998951, 9998952, 9998953, 9998954, 9998956, 9998957, 9998958, 9998959, 9998960, 9998961, 9998962, 9998963, 9998964, 9998965, 9998966, 9998967, 9998968, 9998969, 9998970, 9998971, 9998972, 9998973, 9998974, 9998975, 9998976, 9998977, 9998978, 9998979, 9998980, 9998981, 9998982, 9998983, 9998984, 9998985, 9998986, 9998987, 9998988, 9998989, 9998990, 9998991, 9998992, 9998993, 9998994, 9998995, 9998996, 9998998, 9998999, 9999000, 9999001, 9999002, 9999003, 9999004, 9999005, 9999006, 9999007, 9999008, 9999009, 9999010, 9999011, 9999012, 9999013, 9999014, 9999015, 9999016, 9999017, 9999018, 9999019, 9999020, 9999021, 9999022, 9999023, 9999024, 9999025, 9999026, 9999027, 9999028, 9999029, 9999030, 9999031, 9999032, 9999033, 9999034, 9999035, 9999036, 9999037, 9999038, 9999039, 9999041, 9999042, 9999043, 9999044, 9999045, 9999046, 9999047, 9999048, 9999049, 9999050, 9999051, 9999052, 9999053, 9999054, 9999055, 9999056, 9999057, 9999058, 9999059, 9999060, 9999061, 9999062, 9999063, 9999064, 9999065, 9999066, 9999067, 9999068, 9999069, 9999070, 9999071, 9999072, 9999073, 9999074, 9999075, 9999076, 9999077, 9999078, 9999079, 9999080, 9999081, 9999082, 9999084, 9999085, 9999086, 9999087, 9999088, 9999089, 9999090, 9999091, 9999092, 9999093, 9999094, 9999095, 9999096, 9999097, 9999098, 9999099, 9999100, 9999101, 9999102, 9999103, 9999104, 9999105, 9999106, 9999107, 9999108, 9999109, 9999110, 9999111, 9999112, 9999113, 9999114, 9999115, 9999116, 9999117, 9999118, 9999119, 9999120, 9999121, 9999122, 9999123, 9999124, 9999126, 9999127, 9999128, 9999129, 9999130, 9999131, 9999132, 9999133, 9999134, 9999135, 9999136, 9999137, 9999138, 9999139, 9999140, 9999141, 9999142, 9999143, 9999144, 9999145, 9999146, 9999147, 9999148, 9999149, 9999150, 9999151, 9999152, 9999153, 9999154, 9999155, 9999156, 9999157, 9999158, 9999159, 9999160, 9999161, 9999162, 9999163, 9999164, 9999165, 9999166, 9999167, 9999169, 9999170, 9999171, 9999172, 9999173, 9999174, 9999175, 9999176, 9999177, 9999178, 9999179, 9999180, 9999181, 9999182, 9999183, 9999184, 9999185, 9999186, 9999187, 9999188, 9999189, 9999190, 9999191, 9999192, 9999193, 9999194, 9999195, 9999196, 9999197, 9999198, 9999199, 9999200, 9999201, 9999202, 9999203, 9999204, 9999205, 9999206, 9999207, 9999208, 9999209, 9999210, 9999212, 9999213, 9999214, 9999215, 9999216, 9999217, 9999218, 9999219, 9999220, 9999221, 9999222, 9999223, 9999224, 9999225, 9999226, 9999227, 9999228, 9999229, 9999230, 9999231, 9999232, 9999233, 9999234, 9999235, 9999236, 9999237, 9999238, 9999239, 9999240, 9999241, 9999242, 9999243, 9999244, 9999245, 9999246, 9999247, 9999248, 9999249, 9999250, 9999251, 9999252, 9999254, 9999255, 9999256, 9999257, 9999258, 9999259, 9999260, 9999261, 9999262, 9999263, 9999264, 9999265, 9999266, 9999267, 9999268, 9999269, 9999270, 9999271, 9999272, 9999273, 9999274, 9999275, 9999276, 9999277, 9999278, 9999279, 9999280, 9999281, 9999282, 9999283, 9999284, 9999285, 9999286, 9999287, 9999288, 9999289, 9999290, 9999291, 9999292, 9999293, 9999294, 9999295, 9999297, 9999298, 9999299, 9999300, 9999301, 9999302, 9999303, 9999304, 9999305, 9999306, 9999307, 9999308, 9999309, 9999310, 9999311, 9999312, 9999313, 9999314, 9999315, 9999316, 9999317, 9999318, 9999319, 9999320, 9999321, 9999322, 9999323, 9999324, 9999325, 9999326, 9999327, 9999328, 9999329, 9999330, 9999331, 9999332, 9999333, 9999334, 9999335, 9999336, 9999337, 9999338, 9999340, 9999341, 9999342, 9999343, 9999344, 9999345, 9999346, 9999347, 9999348, 9999349, 9999350, 9999351, 9999352, 9999353, 9999354, 9999355, 9999356, 9999357, 9999358, 9999359, 9999360, 9999361, 9999362, 9999363, 9999364, 9999365, 9999366, 9999367, 9999368, 9999369, 9999370, 9999371, 9999372, 9999373, 9999374, 9999375, 9999376, 9999377, 9999378, 9999379, 9999380, 9999382, 9999383, 9999384, 9999385, 9999386, 9999387, 9999388, 9999389, 9999390, 9999391, 9999392, 9999393, 9999394, 9999395, 9999396, 9999397, 9999398, 9999399, 9999400, 9999401, 9999402, 9999403, 9999404, 9999405, 9999406, 9999407, 9999408, 9999409, 9999410, 9999411, 9999412, 9999413, 9999414, 9999415, 9999416, 9999417, 9999418, 9999419, 9999420, 9999421, 9999422, 9999423, 9999425, 9999426, 9999427, 9999428, 9999429, 9999430, 9999431, 9999432, 9999433, 9999434, 9999435, 9999436, 9999437, 9999438, 9999439, 9999440, 9999441, 9999442, 9999443, 9999444, 9999445, 9999446, 9999447, 9999448, 9999449, 9999450, 9999451, 9999452, 9999453, 9999454, 9999455, 9999456, 9999457, 9999458, 9999459, 9999460, 9999461, 9999462, 9999463, 9999464, 9999465, 9999466, 9999468, 9999469, 9999470, 9999471, 9999472, 9999473, 9999474, 9999475, 9999476, 9999477, 9999478, 9999479, 9999480, 9999481, 9999482, 9999483, 9999484, 9999485, 9999486, 9999487, 9999488, 9999489, 9999490, 9999491, 9999492, 9999493, 9999494, 9999495, 9999496, 9999497, 9999498, 9999499, 9999500, 9999501, 9999502, 9999503, 9999504, 9999505, 9999506, 9999507, 9999508, 9999510, 9999511, 9999512, 9999513, 9999514, 9999515, 9999516, 9999517, 9999518, 9999519, 9999520, 9999521, 9999522, 9999523, 9999524, 9999525, 9999526, 9999527, 9999528, 9999529, 9999530, 9999531, 9999532, 9999533, 9999534, 9999535, 9999536, 9999537, 9999538, 9999539, 9999540, 9999541, 9999542, 9999543, 9999544, 9999545, 9999546, 9999547, 9999548, 9999549, 9999550, 9999551, 9999553, 9999554, 9999555, 9999556, 9999557, 9999558, 9999559, 9999560, 9999561, 9999562, 9999563, 9999564, 9999565, 9999566, 9999567, 9999568, 9999569, 9999570, 9999571, 9999572, 9999573, 9999574, 9999575, 9999576, 9999577, 9999578, 9999579, 9999580, 9999581, 9999582, 9999583, 9999584, 9999585, 9999586, 9999587, 9999588, 9999589, 9999590, 9999591, 9999592, 9999593, 9999594, 9999596, 9999597, 9999598, 9999599, 9999600, 9999601, 9999602, 9999603, 9999604, 9999605, 9999606, 9999607, 9999608, 9999609, 9999610, 9999611, 9999612, 9999613, 9999614, 9999615, 9999616, 9999617, 9999618, 9999619, 9999620, 9999621, 9999622, 9999623, 9999624, 9999625, 9999626, 9999627, 9999628, 9999629, 9999630, 9999631, 9999632, 9999633, 9999634, 9999635, 9999636, 9999638, 9999639, 9999640, 9999641, 9999642, 9999643, 9999644, 9999645, 9999646, 9999647, 9999648, 9999649, 9999650, 9999651, 9999652, 9999653, 9999654, 9999655, 9999656, 9999657, 9999658, 9999659, 9999660, 9999661, 9999662, 9999663, 9999664, 9999665, 9999666, 9999667, 9999668, 9999669, 9999670, 9999671, 9999672, 9999673, 9999674, 9999675, 9999676, 9999677, 9999678, 9999679, 9999681, 9999682, 9999683, 9999684, 9999685, 9999686, 9999687, 9999688, 9999689, 9999690, 9999691, 9999692, 9999693, 9999694, 9999695, 9999696, 9999697, 9999698, 9999699, 9999700, 9999701, 9999702, 9999703, 9999704, 9999705, 9999706, 9999707, 9999708, 9999709, 9999710, 9999711, 9999712, 9999713, 9999714, 9999715, 9999716, 9999717, 9999718, 9999719, 9999720, 9999721, 9999722, 9999724, 9999725, 9999726, 9999727, 9999728, 9999729, 9999730, 9999731, 9999732, 9999733, 9999734, 9999735, 9999736, 9999737, 9999738, 9999739, 9999740, 9999741, 9999742, 9999743, 9999744, 9999745, 9999746, 9999747, 9999748, 9999749, 9999750, 9999751, 9999752, 9999753, 9999754, 9999755, 9999756, 9999757, 9999758, 9999759, 9999760, 9999761, 9999762, 9999763, 9999764, 9999766, 9999767, 9999768, 9999769, 9999770, 9999771, 9999772, 9999773, 9999774, 9999775, 9999776, 9999777, 9999778, 9999779, 9999780, 9999781, 9999782, 9999783, 9999784, 9999785, 9999786, 9999787, 9999788, 9999789, 9999790, 9999791, 9999792, 9999793, 9999794, 9999795, 9999796, 9999797, 9999798, 9999799, 9999800, 9999801, 9999802, 9999803, 9999804, 9999805, 9999806, 9999807, 9999809, 9999810, 9999811, 9999812, 9999813, 9999814, 9999815, 9999816, 9999817, 9999818, 9999819, 9999820, 9999821, 9999822, 9999823, 9999824, 9999825, 9999826, 9999827, 9999828, 9999829, 9999830, 9999831, 9999832, 9999833, 9999834, 9999835, 9999836, 9999837, 9999838, 9999839, 9999840, 9999841, 9999842, 9999843, 9999844, 9999845, 9999846, 9999847, 9999848, 9999849, 9999850, 9999852, 9999853, 9999854, 9999855, 9999856, 9999857, 9999858, 9999859, 9999860, 9999861, 9999862, 9999863, 9999864, 9999865, 9999866, 9999867, 9999868, 9999869, 9999870, 9999871, 9999872, 9999873, 9999874, 9999875, 9999876, 9999877, 9999878, 9999879, 9999880, 9999881, 9999882, 9999883, 9999884, 9999885, 9999886, 9999887, 9999888, 9999889, 9999890, 9999891, 9999892, 9999894, 9999895, 9999896, 9999897, 9999898, 9999899, 9999900, 9999901, 9999902, 9999903, 9999904, 9999905, 9999906, 9999907, 9999908, 9999909, 9999910, 9999911, 9999912, 9999913, 9999914, 9999915, 9999916, 9999917, 9999918, 9999919, 9999920, 9999921, 9999922, 9999923, 9999924, 9999925, 9999926, 9999927, 9999928, 9999929, 9999930, 9999931, 9999932, 9999933, 9999934, 9999935, 9999937, 9999938, 9999939, 9999940, 9999941, 9999942, 9999943, 9999944, 9999945, 9999946, 9999947, 9999948, 9999949, 9999950, 9999951, 9999952, 9999953, 9999954, 9999955, 9999956, 9999957, 9999958, 9999959, 9999960, 9999961, 9999962, 9999963, 9999964, 9999965, 9999966, 9999967, 9999968, 9999969, 9999970, 9999971, 9999972, 9999973, 9999974, 9999975, 9999976, 9999977, 9999978, 9999980, 9999981, 9999982, 9999983, 9999984, 9999985, 9999986, 9999987, 9999988, 9999989, 9999990, 9999991, 9999992, 9999993, 9999994, 9999995, 9999996, 9999997, 9999998, 9999999, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 55, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129, 130, 131, 132, 133, 134, 135, 136, 137, 138, 139, 140, 141, 142, 143, 144, 145, 146, 147, 148, 149, 150, 151, 152, 153, 154, 155, 156, 157, 158, 159, 160, 161, 162, 163, 164, 165, 166, 167, 168, 169, 170, 171, 172, 173, 174, 175, 176, 177, 178, 179, 180, 181, 182, 183, 184, 185, 186, 187, 188, 189, 190, 191, 192, 193, 194, 195, 196, 197, 198, 199, 200, 201, 202, 203, 204, 205, 206, 207, 208, 209, 210, 211, 212, 213, 214, 215, 216, 217, 218, 219, 220, 221, 222, 223, 224, 225, 226, 227, 228, 229, 230, 231, 232, 233, 234, 235, 236, 237, 238, 239, 240, 241, 242, 243, 244, 245, 246, 247, 248, 249, 250, 251, 252, 253, 254, 255, 256, 257, 258, 259, 260, 261, 262, 263, 264, 265, 266, 267, 268, 269, 270, 271, 272, 273, 274, 275, 276, 277, 278, 279, 280, 281, 282, 283, 284, 285, 286, 287, 288, 289, 290, 291, 292, 293, 294, 295, 296, 297, 298, 299, 300, 301, 302, 303, 304, 305, 306, 307, 308, 309, 310, 311, 312, 313, 314, 315, 316, 317, 318, 319, 320, 321, 322, 323, 324, 325, 326, 327, 328, 329, 330, 331, 332, 333, 334, 335, 336, 337, 338, 339, 340, 341, 342, 343, 344, 345, 346, 347, 348, 349, 350, 351, 352, 353, 354, 355, 356, 357, 358, 359, 360, 361, 362, 363, 364, 365, 366, 367, 368, 369, 370, 371, 372, 373, 374, 375, 376, 377, 378, 379, 380, 381, 382, 383, 384, 385, 386, 387, 388, 389, 390, 391, 392, 393, 394, 395, 396, 397, 398, 399, 400, 401, 402, 403, 404, 405, 406, 407, 408, 409, 410, 411, 412, 413, 414, 415, 416, 417, 418, 419, 420, 421, 422, 423, 424, 425, 426, 427, 428, 429, 430, 431, 432, 433, 434, 435, 436, 437, 438, 439, 440, 441, 442, 443, 444, 445, 446, 447, 448, 449, 450, 451, 452, 453, 454, 455, 456, 457, 458, 459, 460, 461, 462, 463, 464, 465, 466, 467, 468, 469, 470, 471, 472, 473, 474, 475, 476, 477, 478, 479, 480, 481, 482, 483, 484, 485, 486, 487, 488, 489, 490, 491, 492, 493, 494, 495, 496, 497, 498, 499, 500, 501, 502, 503, 504, 505, 506, 507, 508, 509, 510, 511, 512, 513, 514, 515, 516, 517, 518, 519, 520, 521, 522, 523, 524, 525, 526, 527, 528, 529, 530, 531, 532, 533, 534, 535, 536, 537, 538, 539, 540, 541, 542, 543, 544, 545, 546, 547, 548, 549, 550, 551, 552, 553, 554, 555, 556, 557, 558, 559, 560, 561, 562, 563, 564, 565, 566, 567, 568, 569, 570, 571, 572, 573, 574, 575, 576, 577, 578, 579, 580, 581, 582, 583, 584, 585, 586, 587, 588, 589, 590, 591, 592, 593, 594, 595, 596, 597, 598, 599, 600, 601, 602, 603, 604, 605, 606, 607, 608, 609, 610, 611, 612, 613, 614, 615, 616, 617, 618, 619, 620, 621, 622, 623, 624, 625, 626, 627, 628, 629, 630, 631, 632, 633, 634, 635, 636, 637, 638, 639, 640, 641, 642, 643, 644, 645, 646, 647, 648, 649, 650, 651, 652, 653, 654, 655, 656, 657, 658, 659, 660, 661, 662, 663, 664, 665, 666, 667, 668, 669, 670, 671, 672, 672, 674, 674, 675, 676, 678, 679, 680, 681, 682, 682, 683, 684, 685, 686, 687, 688, 689, 697, 698, 699, 700, 701, 702, 703, 704, 705, 706, 707, 708, 709, 710, 711, 712, 713, 714, 715, 716, 717, 718, 719, 720, 721, 722, 723, 724, 725, 726, 727, 728, 729, 730, 731, 732, 733, 734, 735, 736, 737, 738, 739, 740, 741, 742, 743, 744, 745, 746, 747, 748, 749, 750, 751, 752, 753, 754, 755, 756, 757, 758, 759, 760, 761, 762, 763, 764, 765, 766, 767, 768, 769, 770, 771, 772, 773, 774, 775, 776, 777, 778, 779, 780, 781, 782, 783, 784, 785, 786, 787, 788, 789, 790, 791, 792, 793, 794, 795, 796, 797, 798, 799, 800, 801, 802, 803, 804, 805, 806, 807, 808, 809, 810, 811, 812, 813, 814, 815, 816, 817, 818, 819, 820, 821, 822, 823, 824, 825, 826, 827, 828, 829, 830, 831, 832, 833, 834, 835, 836, 837, 838, 839, 840, 841, 842, 843, 844, 845, 846, 847, 848, 849, 850, 851, 852, 853, 854, 855, 856, 857, 858, 859, 860, 861, 862, 863, 864, 865, 866, 867, 868, 869, 870, 871, 872, 873, 874, 875, 876, 877, 878, 879, 880, 881, 882, 883, 884, 885, 886, 887, 888, 889, 890, 891, 892, 893, 894, 895, 896, 897, 898, 899, 900, 901, 902, 903, 904, 905, 906, 907, 908, 909, 910, 911, 912, 913, 914, 915, 916, 917, 918, 919, 920, 921, 922, 923, 924, 925, 926, 927, 928, 929, 930, 931, 932, 933, 934, 935, 936, 937, 938, 939, 940, 941, 942, 943, 944, 945, 946, 947, 948, 949, 950, 951, 952, 953, 954, 955, 956, 957, 958, 959, 960, 961, 962, 963, 964, 965, 966, 967, 968, 969, 970, 971, 972, 973, 974, 975, 976, 977, 978, 979, 980, 981, 982, 983, 984, 985, 986, 987, 988, 989, 990, 991, 992, 993, 994, 995, 996, 997, 998, 999, 1000, 1001, 1002, 1003, 1004, 1005, 1006, 1007, 1008, 1009, 1010, 1011, 1012, 1013, 1014, 1015, 1016, 1017, 1018, 1019, 1020, 1021, 1022, 1023, 1024, 1025, 1026, 1027, 1028, 1029, 1030, 1031, 1032, 1033, 1034, 1035, 1036, 1037, 1038, 1039, 1040, 1041, 1042, 1043, 1044, 1045, 1046, 1047, 1048, 1049, 1050, 1051, 1052, 1053, 1054, 1055, 1056, 1057, 1058, 1059, 1060, 1061, 1062, 1063, 1064, 1065, 1066, 1067, 1068, 1069, 1070, 1071, 1072, 1073, 1074, 1075, 1076, 1077, 1078, 1079, 1080, 1081, 1082, 1083, 1084, 1085, 1086, 1087, 1088, 1089, 1090, 1091, 1092, 1093, 1094, 1095, 1096, 1097, 1098, 1099, 1100, 1101, 1102, 1103, 1104, 1105, 1106, 1107, 1108, 1109, 1110, 1111, 1112, 1113, 1114, 1115, 1116, 1117, 1118, 1119, 1120, 1121, 1122, 1123, 1124, 1125, 1126, 1127, 1128, 1129, 1130, 1131, 1132, 1133, 1134, 1135, 1136, 1137, 1138, 1139, 1140, 1141, 1142, 1143, 1144, 1145, 1146, 1147, 1148, 1149, 1150, 1151, 1152, 1153, 1154, 1155, 1156, 1157, 1158, 1159, 1160, 1161, 1162, 1163, 1164, 1165, 1166, 1167, 1168, 1169, 1170, 1171, 1172, 1173, 1174, 1175, 1176, 1177, 1178, 1179, 1180, 1181, 1182, 1183, 1184, 1185, 1186, 1187, 1188, 1189, 1190, 1191, 1192, 1193, 1194, 1195, 1196, 1197, 1198, 1199, 1200, 1201, 1202, 1203, 1204, 1205, 1206, 1207, 1208, 1209, 1210, 1211, 1212, 1213, 1214, 1215, 1216, 1217, 1218, 1219, 1220, 1221, 1222, 1223, 1224, 1225, 1226, 1227, 1228, 1229, 1230, 1231, 1232, 1233, 1234, 1235, 1236, 1237, 1238, 1239, 1240, 1241, 1242, 1243, 1244, 1245, 1246, 1247, 1248, 1249, 1250, 1251, 1252, 1253, 1254, 1255, 1256, 1257, 1258, 1259, 1260, 1261, 1262, 1263, 1264, 1265, 1266, 1267, 1268, 1269, 1270, 1271, 1272, 1273, 1274, 1275, 1276, 1277, 1278, 1279, 1280, 1281, 1282, 1283, 1284, 1285, 1286, 1287, 1288, 1289, 1290, 1291, 1292, 1293, 1294, 1295, 1296, 1297, 1298, 1299, 1300, 1301, 1302, 1303, 1304, 1305, 1306, 1307, 1308, 1309, 1310, 1311, 1312, 1313, 1314, 1315, 1316, 1317, 1318, 1319, 1320, 1321, 1322, 1323, 1324, 1325, 1326, 1327, 1328, 1329, 1330, 1331, 1332, 1333, 1334, 1335, 1336, 1337, 1338, 1339, 1340, 1341, 1342, 1343, 1344, 1345, 1346, 1347, 1348, 1349, 1350, 1351, 1352, 1353, 1354, 1355, 1356, 1357, 1358, 1359, 1360, 1361, 1362, 1363, 1364, 1365, 1366, 1367, 1368, 1369, 1370, 1371, 1372, 1373, 1374, 1375, 1376, 1377, 1378, 1379, 1380, 1381, 1382, 1383, 1384, 1385, 1386, 1387, 1388, 1389, 1390, 1391, 1392, 1393, 1394, 1395, 1396, 1397, 1398, 1399, 1400, 1401, 1402, 1403, 1404, 1405, 1406, 1407, 1408, 1409, 1410, 1411, 1412, 1413, 1414, 1415, 1416, 1417, 1418, 1419, 1420, 1421, 1422, 1423, 1424, 1425, 1426, 1427, 1428, 1429, 1430, 1431, 1432, 1433, 1434, 1435, 1436, 1437, 1438, 1439, 1440, 1441, 1442, 1443, 1444, 1445, 1446, 1447, 1448, 1449, 1450, 1451, 1452, 1453, 1454, 1455, 1456, 1457, 1458, 1459, 1460, 1461, 1462, 1463, 1464, 1465, 1466, 1467, 1468, 1469, 1470, 1471, 1472, 1473, 1474, 1475, 1476, 1477, 1478, 1479, 1480, 1481, 1482, 1483, 1484, 1485, 1486, 1487, 1488, 1489, 1490, 1491, 1492, 1493, 1494, 1495, 1496, 1497, 1498, 1499, 1500, 1501, 1502, 1503, 1504, 1505, 1506, 1507, 1508, 1509, 1510, 1511, 1512, 1513, 1514, 1515, 1516, 1517, 1518, 1519, 1520, 1521, 1522, 1523, 1524, 1525, 1526, 1527, 1528, 1529, 1530, 1531, 1532, 1533, 1534, 1535, 1536, 1537, 1538, 1539, 1540, 1541, 1542, 1543, 1544, 1545, 1546, 1547, 1548, 1549, 1550, 1551, 1552, 1553, 1554, 1555, 1556, 1557, 1558, 1559, 1560, 1561, 1562, 1563, 1564, 1565, 1566, 1567, 1568, 1569, 1570, 1571, 1572, 1573, 1574, 1575, 1575, 1579, 1580, 1581, 1582, 1583, 1584, 1585, 1586, 1587, 1588, 1589, 1590, 1591, 1592, 1593, 1594, 1595, 1596, 1597, 1598, 1599, 1600, 1601, 1602, 1603, 1604, 1605, 1606, 1607, 1608, 1609, 1610, 1611, 1612, 1613, 1614, 1615, 1616, 1617, 1618, 1619, 1620, 1621, 1622, 1623, 1624, 1625, 1626, 1627, 1628, 1629, 1630, 1631, 1632, 1633, 1634, 1635, 1636, 1637, 1638, 1639, 1640, 1641, 1642, 1643, 1644, 1645, 1646, 1647, 1648, 1649, 1650, 1651, 1652, 1653, 1654, 1655, 1656, 1657, 1658, 1659, 1660, 1661, 1662, 1663, 1664, 1665, 1666, 1667, 1668, 1669, 1670, 1671, 1672, 1673, 1674, 1675, 1676, 1677, 1678, 1679, 1680, 1681, 1682, 1683, 1684, 1685, 1686, 1687, 1688, 1689, 1690, 1691, 1692, 1693, 1694, 1695, 1696, 1697, 1698, 1699, 1700, 1701, 1702, 1703, 1704, 1705, 1706, 1707, 1708, 1709, 1710, 1711, 1712, 1713, 1714, 1715, 1716, 1717, 1718, 1719, 1720, 1721, 1722, 1723, 1724, 1725, 1726, 1727, 1728, 1729, 1730, 1731, 1732, 1733, 1734, 1735, 1736, 1737, 1738, 1739, 1740, 1741, 1742, 1743, 1744, 1745, 1746, 1747, 1748, 1749, 1750, 1751, 1752, 1753, 1754, 1755, 1756, 1757, 1758, 1759, 1760, 1761, 1762, 1763, 1764, 1765, 1766, 1767, 1768, 1769, 1770, 1771, 1772, 1773, 1774, 1775, 1776, 1777, 1778, 1779, 1780, 1781, 1782, 1783, 1784, 1785, 1786, 1787, 1788, 1789, 1790, 1791, 1792, 1793, 1794, 1795, 1796, 1797, 1798, 1799, 1800, 1801, 1802, 1803, 1804, 1805, 1806, 1807, 1808, 1809, 1810, 1811, 1812, 1813, 1814, 1815, 1816, 1817, 1818, 1819, 1820, 1821, 1822, 1823, 1824, 1825, 1826, 1827, 1828, 1829, 1830, 1831, 1832, 1833, 1834, 1835, 1836, 1837, 1838, 1839, 1840, 1841, 1842, 1843, 1844, 1845, 1846, 1847, 1848, 1849, 1850, 1851, 1852, 1853, 1854, 1855, 1856, 1857, 1858, 1859, 1860, 1861, 1862, 1863, 1864, 1865, 1866, 1867, 1868, 1869, 1870, 1871, 1872, 1873, 1874, 1875, 1876, 1877, 1878, 1879, 1880, 1881, 1882, 1883, 1884, 1885, 1886, 1887, 1888, 1889, 1890, 1891, 1892, 1893, 1894, 1895, 1896, 1897, 1898, 1899, 1900, 1901, 1902, 1903, 1904, 1905, 1906, 1907, 1908, 1909, 1910, 1911, 1912, 1913, 1914, 1915, 1916, 1917, 1918, 1919, 1920, 1921, 1922, 1923, 1924, 1925, 1926, 1927, 1928, 1929, 1930, 1931, 1932, 1933, 1934, 1935, 1936, 1937, 1938, 1939, 1940, 1941, 1942, 1943, 1944, 1945, 1946, 1947, 1948, 1949, 1950, 1951, 1952, 1953, 1954, 1955, 1956, 1957, 1958, 1959, 1960, 1961, 1962, 1963, 1964, 1965, 1966, 1967, 1968, 1969, 1970, 1971, 1972, 1973, 1974, 1975, 1976, 1977, 1978, 1979, 1980, 1981, 1982, 1983, 1984, 1985, 1986, 1987, 1988, 1989, 1990, 1991, 1992, 1993, 1994, 1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010, 2011, 2012, 2013, 2014, 2015, 2016, 2017, 2018, 2019, 2020, 2021, 2022, 2023, 2024, 2025, 2026, 2027, 2028, 2029, 2030, 2031, 2032, 2033, 2034, 2035, 2036, 2037, 2038, 2039, 2040, 2041, 2042, 2043, 2044, 2045, 2046, 2047, 2048, 2049, 2050, 2051, 2052, 2053, 2054, 2055, 2056, 2057, 2058, 2059, 2060, 2061, 2062, 2063, 2064, 2065, 2066, 2067, 2068, 2069, 2070, 2071, 2072, 2073, 2074, 2075, 2076, 2077, 2078, 2079, 2080, 2081, 2082, 2083, 2084, 2085, 2086, 2087, 2088, 2089, 2090, 2091, 2092, 2093, 2094, 2095, 2096, 2097, 2098, 2099, 2100, 2101, 2102, 2103, 2104, 2105, 2106, 2107, 2108, 2109, 2110, 2111, 2112, 2113, 2114, 2115, 2116, 2117, 2118, 2119, 2120, 2121, 2122, 2123, 2124, 2125, 2126, 2127, 2128, 2129, 2130, 2131, 2132, 2133, 2134, 2135, 2136, 2137, 2138, 2139, 2140, 2141, 2142, 2143, 2144, 2145, 2146, 2147, 2148, 2149, 2150, 2151, 2152, 2153, 2154, 2155, 2156, 2157, 2158, 2159, 2160, 2161, 2162, 2163, 2164, 2165, 2166, 2167, 2168, 2169, 2170, 2171, 2172, 2173, 2174, 2175, 2176, 2177, 2178, 2178, 2179, 2180, 2182, 2183, 2184, 2185, 2186, 2187, 2188, 2189, 2190, 2191, 2192, 2193, 2194, 2195, 2196, 2197, 2198, 2199, 2200, 2201, 2202, 2203, 2204, 2205, 2206, 2207, 2208, 2209, 2210, 2211, 2212, 2213, 2214, 2215, 2216, 2217, 2218, 2219, 2220, 2221, 2222, 2223, 2224, 2225, 2226, 2227, 2228, 2229, 2230, 2231, 2232, 2233, 2234, 2235, 2236, 2237, 2238, 2239, 2240, 2241, 2242, 2243, 2244, 2245, 2246, 2247, 2248, 2249, 2250, 2251, 2252, 2253, 2254, 2255, 2256, 2257, 2258, 2259, 2260, 2261, 2262, 2263, 2264, 2265, 2266, 2267, 2268, 2269, 2270, 2271, 2272, 2273, 2274, 2275, 2276, 2277, 2278, 2279, 2280, 2281, 2282, 2283, 2284, 2285, 2286, 2287, 2288, 2289, 2290, 2291, 2292, 2293, 2294, 2295, 2296, 2297, 2298, 2299, 2300, 2301, 2302, 2303, 2304, 2305, 2306, 2307, 2308, 2309, 2310, 2311, 2312, 2313, 2314, 2315, 2316, 2317, 2318, 2319, 2320, 2321, 2322, 2323, 2324, 2325, 2326, 2327, 2328, 2329, 2330, 2331, 2332, 2333, 2334, 2335, 2336, 2337, 2338, 2339, 2340, 2341, 2342, 2343, 2344, 2345, 2346, 2347, 2348, 2349, 2350, 2351, 2352, 2353, 2354, 2354, 2356, 2356, 2357, 2358, 2359, 2359, 2360, 2361, 2361, 2362, 2363, 2364, 2365, 2366, 2367, 2369, 2370, 2371, 2372, 2373, 2374, 2375, 2376, 2377, 2378, 2379, 2380, 2381, 2382, 2383, 2384, 2385, 2386, 2387, 2388, 2389, 2390, 2390, 2391, 2392, 2393, 2394, 2395, 2396, 2397, 2398, 2399, 2400, 2401, 2402, 2403, 2407, 2408, 2409, 2410, 2411, 2412, 2413, 2414, 2415, 2416, 2417, 2418, 2419, 2420, 2421, 2422, 2423, 2424, 2425, 2426, 2427, 2428, 2429, 2430, 2431, 2432, 2433, 2434, 2435, 2436, 2437, 2438, 2439, 2440, 2441, 2442, 2443, 2444, 2445, 2446, 2447, 2448, 2449, 2450, 2451, 2452, 2453, 2454, 2455, 2456, 2457, 2458, 2459, 2460, 2461, 2462, 2463, 2464, 2465, 2466, 2467, 2468, 2469, 2470, 2471, 2472, 2473, 2474, 2475, 2476, 2477, 2478, 2479, 2480, 2481, 2482, 2483, 2484, 2485, 2486, 2487, 2488, 2489, 2490, 2491, 2492, 2493, 2494, 2495, 2496, 2497, 2498, 2499, 2500, 2501, 2502, 2503, 2504, 2505, 2506, 2507, 2508, 2509, 2510, 2511, 2512, 2513, 2514, 2515, 2516, 2517, 2518, 2519, 2520, 2521, 2522, 2523, 2524, 2525, 2526, 2527, 2528, 2529, 2530, 2531, 2532, 2533, 2534, 2535, 2536, 2537, 2538, 2539, 2540, 2541, 2542, 2543, 2544, 2545, 2546, 2547, 2548, 2549, 2550, 2551, 2552, 2553, 2554, 2555, 2556, 2557, 2558, 2558, 2559, 2560, 2562, 2563, 2564, 2565, 2566, 2566, 2567, 2567, 2572, 2573, 2574, 2575, 2576, 2577, 2578, 2579, 2580, 2581, 2582, 2583, 2584, 2585, 2586, 2587, 2588, 2589, 2590, 2591, 2592, 2593, 2594, 2595, 2596, 2597, 2598, 2599, 2600, 2601, 2602, 2603, 2604, 2605, 2606, 2607, 2608, 2609, 2610, 2611, 2612, 2613, 2614, 2615, 2616, 2617, 2618, 2619, 2620, 2621, 2622, 2623, 2624, 2625, 2626, 2627, 2628, 2629, 2630, 2630]

and the output looks great:

Incremental Sequence 1: First = 9998424, Last = 9999999
Incremental Sequence 2: First = 0, Last = 2630
('Sum of differences between first and last values:', 4205)

Unfortunately, the moment I included one of the extraneous values from your first post the script gives an incorrect output:

my_list = [9998424, 9998425, 9998426, 9998427, 20, 9998429, [...]
Incremental Sequence 1: First = 9998424, Last = 9998427
Incremental Sequence 2: First = 20, Last = 9999999
Incremental Sequence 3: First = 0, Last = 2630
('Sum of differences between first and last values:', 10002612)

It's worth noting that the script I posted includes a validity check that filters out the spike values per your original question. The same experiment does not render a false output.

1 Like

Thanks for the update

1 Like