Searched refs:quantize (Results 1 – 13 of 13) sorted by relevance
2 -- dqQuantize.decTest -- decQuad quantize operation --24 -- 2004.03.15 Underflow for quantize is suppressed26 -- [Forked from quantize.decTest 2006.11.25]36 dqqua001 quantize 0 1e0 -> 037 dqqua002 quantize 1 1e0 -> 138 dqqua003 quantize 0.1 1e+2 -> 0E+2 Inexact Rounded39 dqqua005 quantize 0.1 1e+1 -> 0E+1 Inexact Rounded40 dqqua006 quantize 0.1 1e0 -> 0 Inexact Rounded41 dqqua007 quantize 0.1 1e-1 -> 0.142 dqqua008 quantize 0.1 1e-2 -> 0.10[all …]
2 -- ddQuantize.decTest -- decDouble quantize operation --24 -- 2004.03.15 Underflow for quantize is suppressed34 ddqua001 quantize 0 1e0 -> 035 ddqua002 quantize 1 1e0 -> 136 ddqua003 quantize 0.1 1e+2 -> 0E+2 Inexact Rounded37 ddqua005 quantize 0.1 1e+1 -> 0E+1 Inexact Rounded38 ddqua006 quantize 0.1 1e0 -> 0 Inexact Rounded39 ddqua007 quantize 0.1 1e-1 -> 0.140 ddqua008 quantize 0.1 1e-2 -> 0.1041 ddqua009 quantize 0.1 1e-3 -> 0.100[all …]
2 -- quantize.decTest -- decimal quantize operation --24 -- 2004.03.15 Underflow for quantize is suppressed34 quax001 quantize 0 1e0 -> 035 quax002 quantize 1 1e0 -> 136 quax003 quantize 0.1 1e+2 -> 0E+2 Inexact Rounded37 quax005 quantize 0.1 1e+1 -> 0E+1 Inexact Rounded38 quax006 quantize 0.1 1e0 -> 0 Inexact Rounded39 quax007 quantize 0.1 1e-1 -> 0.140 quax008 quantize 0.1 1e-2 -> 0.1041 quax009 quantize 0.1 1e-3 -> 0.100[all …]
287 ddcan401 quantize #6e38ff3ffff3fcff 1 -> #6e38ff3fcff3fcff288 ddcan402 quantize #6e38ff3fcff3fdff 0 -> #6e38ff3fcff3fcff289 ddcan403 quantize #7880000000000000 Inf -> #7800000000000000290 ddcan404 quantize #7802000000000000 -Inf -> #7800000000000000291 ddcan410 quantize #7c03ff3fcff3fcff 1 -> #7c00ff3fcff3fcff292 ddcan411 quantize #7c03ff3fcff3fcff 1 -> #7c00ff3fcff3fcff293 ddcan412 quantize #7c40ff3fcff3fcff 1 -> #7c00ff3fcff3fcff294 ddcan413 quantize #7c40ff3fcff3fcff 1 -> #7c00ff3fcff3fcff295 ddcan414 quantize #7e00ffffcff3fcff 1 -> #7c00ff3fcff3fcff Invalid_operation296 ddcan415 quantize #7e00ffffcff3fcff 1 -> #7c00ff3fcff3fcff Invalid_operation[all …]
310 dqcan401 quantize #ee080ff3fcff3fcff3fffffffff3fcff 0 -> #ee080ff3fcff3fcff3fcff3fcff3fcff311 dqcan402 quantize #ee080ff3fffffffffffcff3fcff3fcff 0 -> #ee080ff3fcff3fcff3fcff3fcff3fcff312 dqcan403 quantize #78800000000000000000000000000000 Inf -> #78000000000000000000000000000000313 dqcan404 quantize #78020000000000000000000000000000 -Inf -> #78000000000000000000000000000000314 dqcan410 quantize #7c080ff3fcff3fcff3fcff3fcff3fcff 1 -> #7c000ff3fcff3fcff3fcff3fcff3fcff315 dqcan411 quantize #fc000ff3fcfffffff3fcff3fcff3fcff 1 -> #fc000ff3fcff3fcff3fcff3fcff3fcff316 dqcan412 quantize #7e100ff3fcff3fcff3fcff3fcff3fcff 1 -> #7c000ff3fcff3fcff3fcff3fcff3fcff Inva…317 dqcan413 quantize #fe000ff3fcff3fcff3ffffffcff3fcff 1 -> #fc000ff3fcff3fcff3fcff3fcff3fcff Inva…
60 dectest: quantize
25 -- quantize, so those are assumed to have been tested.80 -- next two would be NaN using quantize(x, 0)
25 -- quantize, so those are assumed to have been tested extensively87 -- next two would be NaN using quantize(x, 0)
24 -- 2004.03.15 Underflow for quantize is suppressed409 -- quantize.decTest. Here's a critical one.]
1632 self.assertEqual(Decimal(1234).quantize(100),1633 Decimal(1234).quantize(Decimal(100)))2100 d = c.quantize(Decimal(1), Decimal(2))2101 self.assertEqual(c.quantize(1, 2), d)2102 self.assertEqual(c.quantize(Decimal(1), 2), d)2103 self.assertEqual(c.quantize(1, Decimal(2)), d)2104 self.assertRaises(TypeError, c.quantize, '1', 2)2105 self.assertRaises(TypeError, c.quantize, 1, '2')
2394 def quantize(self, exp, rounding=None, context=None, watchexp=True): member in Decimal5017 def quantize(self, a, b): member in Context5073 return a.quantize(b, context=self)