Math Hierarchy; Math Errors; Making Comparisons Work - Agilent Technologies 3458A User Manual

Math Hierarchy

Math Errors

Making Comparisons
272 Chapter 7 BASIC Language for the 3458A
Function/Argument
ROTATE(X,displacement) Returns an integer obtained by rotating the argument a
SHIFT(X,displacement)
*
If the displacement is positive, rotating or shifting is toward the least significant
bit.
If the displacement is negative, rotating or shifting is toward the most significant
bit.
The 3458A evaluates parenthetical expressions before evaluating any math
functions outside of parentheses. If two or more operations of the same
priority are in the expression, the hierarchy is from left to right
.
Highest Priority
Lowest Priority
When evaluating a math expression, the following errors may occur. The
3458A treats math errors just like any other execution errors. Refer to chapter
3 for more information on handling errors.
If you are making mathematical comparisons between integer numbers, no
special precautions are necessary. However, if you are comparing REAL
Work
numbers, especially those which are the results of calculations, it is possible
that you might run into problems due to rounding and other limitations
inherent in the system. For example, consider the use of the IF...THEN
statement to check for equality in any situation resembling the following
example.
10 OUTPUT 722; "SUB TESTER"
specified number of positions with bit wraparound.*
Returns an integer obtained by rotating the argument a
specified number of positions without bit wraparound.*
Parentheses
Functions: SIN, COS, etc.
Exponentiation
*, /, MOD, DIV
+, –
Relational Operators: <, >, < =, >=, etc.
logical operators: AND, OR, etc.
Error Description
Division by Zero
Real Overflow
Real Underflow
Integer Overflow
Square Root of a Negative Number
Log of a Non-Positive Number
Illegal Real Number
Trig Argument Out of Range
BCD Exponent Too Big
HEX, Octal, or Decimal Argument Error
Meaning