开云体育

It would be helpful if you described the specific frequency span you are

using, the number of segments,

and the frequencies where the "errors" occur. If there is some sort of

bug in the software algorithm,

that would help to see where it originates.

** Hello
I am posting an example. The calibration file (.cal) corresponds to the
end of a 50 m coaxial cable (50 Ω). The frequency sweep range is 1 to 20
Mhz in 20 segments.

Follows the description of each sheet in the .xls file

.cal
---
Raw data from the .cal file converted through a .csv file

Garden 1-20 MHz 20 sec avg 25-6
-------------------------------------
Data from the .cal sheet

Algo
----
* Column A to G: the input data. It is "by hand" that I zoned in yellow
the cells identified as "in error".

* Column I to N: a first search for errors with formulas of the kind
=IF(ABS((D19+D21)/2-D20)>$J1;1;"")
In cell J1: the value of the maximum difference before being declared
abnormal. In my example 0.005 (0.5%) works fine

* Column P to U: precise identification of the cell in error by a formula
like
=IF(AND(K21=1;K20=1;K22=1);1;"")

* Column W to AB: correction of values ??coming from columns A to G with
formulas like
=IF(R13=1;(D12+D14)/2;D13)

* CSV-final
Corrected data formatting. The first three lines are special to take into
account that the algorithm does not know how to process the first three
lines of the original file
This sheet will be saved in .csv in order to produce the corrected .cal
but do not forget, beforehand, to save the workbook if you want to keep
track of the calculations. I added in this sheet the graph of one of the
shortR column to show that the evolution of the values of sinusoidal form

But it is also true that the nanoVNA hardware also has limitations and

boundaries,

and it is likely that the "error" points you are seeing in your

calibration is in fact a true

measurement, due to a condition in the hardware that happens at the

subject frequency. And so such an "error"

** It is very unlikely since in this case, it is a calibration file (.cal)
following the operation (closed, open load). I don't have access to the
built-in average function in nanaovna-saver. I'm just seeing the obvious
errors easily identify in the produced .cal file. The errors that my
algorithm locates are big errors that produce straight line segments in the
graphs with the worst effect. There may be other finer errors that I can't
locate.

--
F1AMM (Fran?ois)

On 7/28/22 11:32 PM, F1AMM wrote:

Hello

I didn't understand the meaning of:

- Number of measurements to average

The NanoVNA makes multiple sweeps, and NanoVNA-Saver averages them.

One thing to remember is that if the sweep is >101 points, it's
actually multiple sweeps strung together.

- Number to discard

I do not know. Maybe it's how many that are farthest from the mean?
(i.e. throwing out outliers)

Looking at SweepWorker.py, that's what it looks like. See below for
the code.

- Common values ??are 3/0, 5/5, 9/4 and 25/6

1/ Could you explain to me what you understand about this subject?

I saw that these parameters were also used during the calibration.

During the calibration sequence (short open load), some measurements
are marred by a gross error (click). These errors, despite a 25/6
filtering, remain present in the produced .cal file. There is nothing
more unpleasant than to find, then, always in the same place, these
errors in the normal measurements using this .cal file


Are these at boundaries of sweeps (i.e. at sample 101, 201, etc.)?

I have to go back, almost by hand, to the raw .cal file to correct
its

errors. I detect errors with Excel and I correct them by doing a
linear extrapolation with the previous value and the next value. The
result is satisfying but not very effective.

My questions
-------------

2/ What is the nature of the current filtering algorithm that leaves
so

much error
I don't know that there is a smoother or filter across the band. My
impression is that it is all point by point.

3/ Could this algorithm be improved?

Probably - there's always room for improvement.

It's tricky doing outlier rejection or filtering though. Consider
measuring a narrow band filter with measurement points far enough
apart that the sequence goes "out of band", "in band", "out of band" -
is the radically different number of the middle sample an outlier, or
the true value.

4/ Has anyone ever come up with a spinner, to run on the raw .cal
file;

to correct these gross errors

73

This is what removes outliers:


def truncate(values: List[List[Tuple]], count: int) -> List[List[Tuple]]:
"""truncate drops extrema from data list if averaging is active"""
keep = len(values) - count
logger.debug("Truncating from %d values to %d", len(values), keep)
if count < 1 or keep < 1:
logger.info("Not doing illegal truncate")
return values
truncated = []
for valueset in np.swapaxes(values, 0, 1).tolist():
avg = complex(*np.average(valueset, 0))
truncated.append(
sorted(valueset,
key=lambda v, a=avg:
abs(a - complex(*v)))[:keep])
return np.swapaxes(truncated, 0, 1).tolist()

On 7/28/22 11:32 PM, F1AMM wrote:

Hello

I didn't understand the meaning of:

- Number of measurements to average

The NanoVNA makes multiple sweeps, and NanoVNA-Saver averages them.

One thing to remember is that if the sweep is >101 points, it's actually
multiple sweeps strung together.

- Number to discard

I do not know. Maybe it's how many that are farthest from the mean?
(i.e. throwing out outliers)

Looking at SweepWorker.py, that's what it looks like. See below for the
code.

- Common values ??are 3/0, 5/5, 9/4 and 25/6

1/ Could you explain to me what you understand about this subject?

I saw that these parameters were also used during the calibration.

During the calibration sequence (short open load), some measurements are
marred by a gross error (click). These errors, despite a 25/6 filtering,
remain present in the produced .cal file. There is nothing more unpleasant
than to find, then, always in the same place, these errors in the normal
measurements using this .cal file


Are these at boundaries of sweeps (i.e. at sample 101, 201, etc.)?

I have to go back, almost by hand, to the raw .cal file to correct its

errors. I detect errors with Excel and I correct them by doing a linear
extrapolation with the previous value and the next value. The result is
satisfying but not very effective.

My questions
-------------

2/ What is the nature of the current filtering algorithm that leaves so

much error
I don't know that there is a smoother or filter across the band. My
impression is that it is all point by point.

3/ Could this algorithm be improved?

Probably - there's always room for improvement.

It's tricky doing outlier rejection or filtering though. Consider
measuring a narrow band filter with measurement points far enough apart
that the sequence goes "out of band", "in band", "out of band" - is the
radically different number of the middle sample an outlier, or the true
value.

4/ Has anyone ever come up with a spinner, to run on the raw .cal file;

to correct these gross errors

73

This is what removes outliers:


def truncate(values: List[List[Tuple]], count: int) -> List[List[Tuple]]:
"""truncate drops extrema from data list if averaging is active"""
keep = len(values) - count
logger.debug("Truncating from %d values to %d", len(values), keep)
if count < 1 or keep < 1:
logger.info("Not doing illegal truncate")
return values
truncated = []
for valueset in np.swapaxes(values, 0, 1).tolist():
avg = complex(*np.average(valueset, 0))
truncated.append(
sorted(valueset,
key=lambda v, a=avg:
abs(a - complex(*v)))[:keep])
return np.swapaxes(truncated, 0, 1).tolist()

Which nano and which firmware?

Do different nano-types or firmware versions make a difference?

On my old nano (H3.2) I had a bump around 6 MHz, but it was a known errror.

Arie PA3A

Hello

I didn't understand the meaning of:

- Number of measurements to average
- Number to discard
- Common values ??are 3/0, 5/5, 9/4 and 25/6

1/ Could you explain to me what you understand about this subject?

I saw that these parameters were also used during the calibration.

During

the calibration sequence (short open load), some measurements are

marred by

a gross error (click). These errors, despite a 25/6 filtering, remain
present in the produced .cal file. There is nothing more unpleasant

than to

find, then, always in the same place, these errors in the normal
measurements using this .cal file

I have to go back, almost by hand, to the raw .cal file to correct its
errors. I detect errors with Excel and I correct them by doing a linear
extrapolation with the previous value and the next value. The result is
satisfying but not very effective.

My questions
-------------

2/ What is the nature of the current filtering algorithm that leaves so
much error
3/ Could this algorithm be improved?
4/ Has anyone ever come up with a spinner, to run on the raw .cal file;

to

correct these gross errors

73
--
F1AMM (Fran?ois)

Hello

I didn't understand the meaning of:

- Number of measurements to average
- Number to discard
- Common values ??are 3/0, 5/5, 9/4 and 25/6

1/ Could you explain to me what you understand about this subject?

I saw that these parameters were also used during the calibration. During
the calibration sequence (short open load), some measurements are marred by
a gross error (click). These errors, despite a 25/6 filtering, remain
present in the produced .cal file. There is nothing more unpleasant than to
find, then, always in the same place, these errors in the normal
measurements using this .cal file

I have to go back, almost by hand, to the raw .cal file to correct its
errors. I detect errors with Excel and I correct them by doing a linear
extrapolation with the previous value and the next value. The result is
satisfying but not very effective.

My questions
-------------

2/ What is the nature of the current filtering algorithm that leaves so
much error
3/ Could this algorithm be improved?
4/ Has anyone ever come up with a spinner, to run on the raw .cal file; to
correct these gross errors

73
--
F1AMM (Fran?ois)

Hello

I didn't understand the meaning of:

- Number of measurements to average
- Number to discard
- Common values ??are 3/0, 5/5, 9/4 and 25/6

1/ Could you explain to me what you understand about this subject?

I saw that these parameters were also used during the calibration. During
the calibration sequence (short open load), some measurements are marred by
a gross error (click). These errors, despite a 25/6 filtering, remain
present in the produced .cal file. There is nothing more unpleasant than to
find, then, always in the same place, these errors in the normal
measurements using this .cal file

I have to go back, almost by hand, to the raw .cal file to correct its
errors. I detect errors with Excel and I correct them by doing a linear
extrapolation with the previous value and the next value. The result is
satisfying but not very effective.

My questions
-------------

2/ What is the nature of the current filtering algorithm that leaves so
much error
3/ Could this algorithm be improved?
4/ Has anyone ever come up with a spinner, to run on the raw .cal file; to
correct these gross errors

73
--
F1AMM (Fran?ois)

-----Message d'origine-----