[SOLVED] Generate Attributes - Strange issue
Hi All,
I've got several basic calculations in the Generate Attributes operator.
I've got one calculation that calculates correctly for some examples but incorrectly for about 5-10% of them.
The calculation is:
if(Amount==(Rate*Units),1,0)
I've got 2 read database operators and a join. The data types for Amount and Rate is "Real". Units is "Int"
For some reason it calculates correctly for most of them but a certain few come up incorrectly. I checked the decimal places of the numbers and they're identical.
Any help would be much appreciated
Cheers,
I've got several basic calculations in the Generate Attributes operator.
I've got one calculation that calculates correctly for some examples but incorrectly for about 5-10% of them.
The calculation is:
if(Amount==(Rate*Units),1,0)
I've got 2 read database operators and a join. The data types for Amount and Rate is "Real". Units is "Int"
For some reason it calculates correctly for most of them but a certain few come up incorrectly. I checked the decimal places of the numbers and they're identical.
Any help would be much appreciated
Cheers,
Find more posts tagged with
Sort by:
1 - 6 of
61
Hi Nils,
The calculation is inconsistent.
For example, using the formula if(Amount==(Rate*Units),1,0)
if(7==(7),1,0) will return 1 in some scenarios and 0 in others without a consistent pattern. It should always be 1 because 7==7........
Is it because of the data types? when you multiply an int by a real number does that cause some sort of issue?
Thanks.
Cheers,
The calculation is inconsistent.
For example, using the formula if(Amount==(Rate*Units),1,0)
if(7==(7),1,0) will return 1 in some scenarios and 0 in others without a consistent pattern. It should always be 1 because 7==7........
Is it because of the data types? when you multiply an int by a real number does that cause some sort of issue?
Thanks.
Cheers,
I figured this out.....when you multiply real by int something weird happens to the data type.
The way I fixed it is by having a "Format Numbers" operator after my first "Generate Attributes" operator to reformat the result. After that I put another "Generate Attributes" operator to perform my calculation.
Not ideal but it does the job.
Thanks anyway
The way I fixed it is by having a "Format Numbers" operator after my first "Generate Attributes" operator to reformat the result. After that I put another "Generate Attributes" operator to perform my calculation.
Not ideal but it does the job.
Thanks anyway
From my first post:
The data types for Amount and Rate is "Real". Units is "Int"
Generate attributes operator 1:
Amount - Real
Rate - Real
Units - Int
Calculation:
Correct_Amount=Rate*Units
Format Numbers Operator:
Correct_Amount
Amount
Subset formatted to numeric
Generate attributes operator 2:
Correct_Amount - Numeric
Amount - Numeric
Rate - Real
Units - Int
Calculation:
if(Correct_Amount==Amount,1,0)
The data types for Amount and Rate is "Real". Units is "Int"
Generate attributes operator 1:
Amount - Real
Rate - Real
Units - Int
Calculation:
Correct_Amount=Rate*Units
Format Numbers Operator:
Correct_Amount
Amount
Subset formatted to numeric
Generate attributes operator 2:
Correct_Amount - Numeric
Amount - Numeric
Rate - Real
Units - Int
Calculation:
if(Correct_Amount==Amount,1,0)
So I think the problem here is, that you test for equality of real values with '=='.
You can have a look here: http://stackoverflow.com/questions/1088216/whats-wrong-with-using-to-compare-floats-in-java
Best,
Nils
You can have a look here: http://stackoverflow.com/questions/1088216/whats-wrong-with-using-to-compare-floats-in-java
Best,
Nils
what do you mean be 'calculating incorrectly'? Could you give an example?
Best,
Nils