I'm sorry if this isn't directly helpful but I thought I could provide my own experience,
because I have downloaded massive amounts of 1min data through the ibapi, and the only types of duplicates I occasionally get are full days, not individual bars.
so if I requested 1m data for two weeks, it sometimes happens that I get 11 days, one being a duplicate.
But none of my sample-cleaning scripts have ever found random duplicate bars within any of the days...?
So, I would certainly check your handling of the data but?I am also not sure to what degree this can differ between clients, so it's also possible that the issue doesn't lie in your script...
?
