I'm trying to perform the following within a view. I'm not getting the
values which I would expect. I'm assuming I have a data type problem where
some precision is being dropped. Could someone give me a heads-up on what
I'm doing wrong? I need full precision.
SUM(((MarketPrice+MarketPriceUp100-(2*MarketPriceUp50))/(MarketPriceUp50*((1
00/10000)^2))/100)
* MarketValue)/ SUM(MarketValue) as [Convexity],
Thanks,
Tom Woods
The Baker Group, LP"Tom Woods" <twoods@.jamesbaker.com> wrote in message
news:OgZCcxTvFHA.2396@.TK2MSFTNGP14.phx.gbl...
> I'm trying to perform the following within a view. I'm not getting the
> values which I would expect. I'm assuming I have a data type problem
> where some precision is being dropped. Could someone give me a heads-up
> on what I'm doing wrong? I need full precision.
>
> SUM(((MarketPrice+MarketPriceUp100-(2*MarketPriceUp50))/(MarketPriceUp50*(
(100/10000)^2))/100)
> * MarketValue)/ SUM(MarketValue) as [Convexity],
>
What are the types of those columns?
Integral types use integer division.
David|||All columns in the formula are 9(12,8).
"David Browne" <davidbaxterbrowne no potted meat@.hotmail.com> wrote in
message news:uyQL80TvFHA.1560@.TK2MSFTNGP09.phx.gbl...
> "Tom Woods" <twoods@.jamesbaker.com> wrote in message
> news:OgZCcxTvFHA.2396@.TK2MSFTNGP14.phx.gbl...
> What are the types of those columns?
> Integral types use integer division.
> David
>|||Hi Tom
This piece of the expression (100/10000) will be carried out as integer
division, and yield 0. If you want decimal results, make at least one of the
operands a decimal (100.0/10000)
HTH
Kalen Delaney, SQL Server MVP
www.solidqualitylearning.com
"Tom Woods" <twoods@.jamesbaker.com> wrote in message
news:OgZCcxTvFHA.2396@.TK2MSFTNGP14.phx.gbl...
> I'm trying to perform the following within a view. I'm not getting the
> values which I would expect. I'm assuming I have a data type problem
> where some precision is being dropped. Could someone give me a heads-up
> on what I'm doing wrong? I need full precision.
>
> SUM(((MarketPrice+MarketPriceUp100-(2*MarketPriceUp50))/(MarketPriceUp50*(
(100/10000)^2))/100)
> * MarketValue)/ SUM(MarketValue) as [Convexity],
> Thanks,
> Tom Woods
> The Baker Group, LP
>|||When I change it to the following I get an error. Could the problem be with
the exponent? Is there a different way of performing the exponential?
((100.0/10000)^2)
BTW, I get the following error when changing 100 to 100.0
--
Microsoft SQL-DMO (ODBC SQLState: 42000)
--
Error 403: Invalid operator for data type. Operator equals boolean XOR, type
equals numeric.
--
OK
--
Thanks,
Tom
"Kalen Delaney" <replies@.public_newsgroups.com> wrote in message
news:usIhD4TvFHA.1032@.TK2MSFTNGP12.phx.gbl...
> Hi Tom
> This piece of the expression (100/10000) will be carried out as integer
> division, and yield 0. If you want decimal results, make at least one of
> the operands a decimal (100.0/10000)
> --
> HTH
> Kalen Delaney, SQL Server MVP
> www.solidqualitylearning.com
>
> "Tom Woods" <twoods@.jamesbaker.com> wrote in message
> news:OgZCcxTvFHA.2396@.TK2MSFTNGP14.phx.gbl...
>
>|||^ operator is bitwise exclusive OR operation for integers. For getting
exponential values as results you might want to try POWER function like:
SELECT POWER( 100.0/10000, 2 )
Anith|||Can you use (1.0/10000) instead of ((100.0/10000)^2)?
Perayu
"Tom Woods" <twoods@.jamesbaker.com> wrote in message
news:uVvmS$TvFHA.596@.TK2MSFTNGP12.phx.gbl...
> When I change it to the following I get an error. Could the problem be
> with the exponent? Is there a different way of performing the
> exponential?
> ((100.0/10000)^2)
>
> BTW, I get the following error when changing 100 to 100.0
> --
> Microsoft SQL-DMO (ODBC SQLState: 42000)
> --
> Error 403: Invalid operator for data type. Operator equals boolean XOR,
> type equals numeric.
> --
> OK
> --
> Thanks,
> Tom
>
> "Kalen Delaney" <replies@.public_newsgroups.com> wrote in message
> news:usIhD4TvFHA.1032@.TK2MSFTNGP12.phx.gbl...
>|||I was able to get it to work by using the POWER function and also changing
the integer values to have precision.
Thanks,
Tom
"Perayu" <yu.he@.state.mn.us.Remove4Replay> wrote in message
news:uPKrC0UvFHA.908@.tk2msftngp13.phx.gbl...
> Can you use (1.0/10000) instead of ((100.0/10000)^2)?
> Perayu
> "Tom Woods" <twoods@.jamesbaker.com> wrote in message
> news:uVvmS$TvFHA.596@.TK2MSFTNGP12.phx.gbl...
>
Subscribe to:
Post Comments (Atom)
No comments:
Post a Comment