Historical & EOD Data Provider & Vendors

Hi CSK,

Thanks.

Decided to make the purchase after getting your reply.
The software is great and it meets my expectation.

Hi hanhao,

"ok good, today the old data came back. Is this yours or SGX's work?"

That was our work. the SGX do not do adjustment to historical data.


"Why is it that the analysts’ data server does not continue the price data in J31 and do adjustment?"

When SGX re-assign a new symbol, the data will be provided only for this new symbol. Old "last traded data before the corporate adjustment" will be provided for the old symbol. At some point the SGX wil remove the old symbol and no data will be provided by SGX. Therefore Analyst's DataServer must use the new symbol going forward.

Hi xpert1974,

We have finally gotten in touch. Thank you.

Just for your info:
We received 3 emails from your Mailer-Daemon on the 24, 25 and 26 Sep about problem delivering our emails to your domian. Our emails were sent on the 22 and 26 Sep.

Hi Hanhao & xpert1974,

We have moved over to the new forum HERE.

Two threads has been created specifically for continuing discussion of CSK's DataServer & TurtleFarm software over HERE.

I am not sure if CSK still accessing this forum, you may want to email him about your query or post at the new forum.

RDgs.. Lion

ok good, today the old data came back. Is this yours or SGX's work?

HI CSK,

As I do not trust yahoo data, I did not commit to buy MLdownloader and have been downloading the SGX daily prices list and making the prices adjustment manually (My initial data back dated to 1980 was downloaded from shareinvestor).

I am interested to get Analyst's DataServer to save myself the trouble of making price adjustment every few months. But I am not sure what to expect from the software. I know it is impossible for you to give a free trial. But what I need is just 1 year worth of OLD data so that I can import to my charting software and test it out before commiting anything.

BTW, can your software separate warrants/rights from the main stocks? When the warrant expired, it is automatically removed? How about stocks that have been suspend for very long (few months)?

For prices adjustment for dividend, what is the policy? Adjust only if it is more than 10%? or? How about capitial reduction?

I have tried senting email to <snipped>@technical-analysis.com twice but did not receive any reply.

I can be contactable at <snipped>@tionghua.net if you think that you are able to provide some help.


Note: edited by CSK on 14-Oct-2007 4:25am to remove email addresses. Prevent spammers harvesting for email addresses.

This post has been edited by csk on Oct 14 2007, 04:24 AM

hey CSK

Today is the first day after a stock split of 1 into 5 for juitian. The old symbol is J31, the new symbol is C8R. Why is it that the analysts’ data server does not continue the price data in J31 and do adjustment?

This post has been edited by hanhao on Sep 27 2007, 12:08 AM

"what are the problems you encountered?"
Still probing for situations where there may be potential problem.

"did yahoo data messed up sgx's data?"
I will not allow this to happen. Database integrity is the highest
of all importance.

"maybe you would like to split them into 2 folders"
This is logistically not possible. The current database is already about
zipped 21 meg to download. Don't want to bloat that. Keeping Yahoo's
SGX data is useless anyway. Yahoo latest intraday snapshot data are to
be appended (and then subsequently over-written with SGX data at EOD)
to the SGX data for continuity of chart analysis. What if Yahoo throw some
data for a stock that is not traded? Perhaps some old prices but with
current date.

A lot of checking and collating.

I don't want to end up with support nightmare if this is rushed and
not tested properly.

what are the problems you encountered? did yahoo data messed up sgx's data? maybe you would like to split them into 2 folders

Firstly, to clarify. This is for intraday snapshot and not for intraday data. They are two different things altogether.

Testing on various things and how Yahoo present them. Most important is - what is available from SGX vs what is available from Yahoo and how to handle the discrepancy properly. How Yahoo might present non-traded stocks and how they may be presented differently from one day to another. There are other things being tested. All in, the aim is not to have Yahoo's data over-writing good data from SGX in the MetaStock database.

hey CSK, how is the intraday data for data analyst coming along?

oh, that's great
meaning that the % advance decline should be correct since untraded stocks are filtered out

The graph is correct. Not sure what you mean by strange. So I make
a guess.

If you are wondering why it should change from day to day, it is
becasue Unch is the number of stocks that is TRADED for that day
and ended UNCHANGED. It DOES NOT include stocks that are NOT
TRADED, whether suspended or not.

For example, from the Stock tab in Analyst's DataServer sorted
in alphabetical order for 27Jul2007:

The first stock, 1st Software, is suspended. It is not traded therefore
is not included in the Unch number.

The 3rd stock, AA Group, is not suspended yet not traded so it is
also not included in the Unch number.

The reasn for Unch to be this way was explained in this thread in
previous postings.

now i am trying to create a % advance decline indicator. Which is


(advance decline spread) / (total number of securities);

where total number of securities = advance + decline + Unch

on metastocks it is

i am getting a strange graph for my total securites. Is this correct?

http://www.imageannex.com/out.php/i1551_1.JPG
(Posted Image)

This post has been edited by hanhao on Jul 29 2007, 12:19 PM

"hey back to this post. i do not understand why composite dollar
volume is not useful"

My personal opinion:

Composite dollar volume of STI was say $100,000,000 on 15Jan200x
for the 30 STI component stocks at that time on that day. Then the
component stocks were changed, and then changed, and then changed,
and then changed, and then changed, and then become 45 component
stocks, and then changed, and then changed, and the component stocks
will be changed at the end of this year when new SGX/SPH/FTSE ones
replcaed the existing ones.

And not to forget if you came to the market recently, the STI today is
not the STII of past years. You may want to check a little on this for
your own knowledge. A little clue, 1997 I think. Or is it 1998?

Now today at this time you plot the STI and the volume (shares or dollar
volume dosen't matter), are they the homogeneous as 15Jan200x?
15Feb200x, 10Sep200x+2, 20Dec200x+5 3Nov200x-3, 3May200x-9?
Are they the same stocks? Are the floating shares the same? Think of
other possible factors. Are they the same? If not what effect on your
analysis?

Now we look at All-Sing Shares. This is wider, no changing of components
stocks at whimps or fancy, to make it look good. Take a look at the DJI,
it is a big joke!.

All-Sing Shares. All Sing $ traded shares are component stocks so
therefore cannot any old how add and delete. Look at this situation: How
many S$ shares in 1998, 1999,2000,2001,2002,2003,2004,2005,2006,
2007,2008,2009,2010 etc. How many listed companies, how many
floating shares. If individual float remains same (not possible) and listed
companies increase, will not composite volume also increase? What effect
on analysis?

Volume analysis on stock index futures is okay, on commodity futures
is okay.

Volume analysis on individual stock may still be okay but not like what
text book teach you. In Singapore, when stocks are hyped up, volume
shoot up. Stocks are hyped up to create a demand to be sold to.
Read Jesse Livermore's book, Reminiscences of a Stock Operator.

The public, the retail, they buy on these hypes (they are the created
demand that are sold to), they get stuck and volume dwindle due to the
stuckist nature prevalent in Singapore. So how to do volume analysis.
Do you notice that OBV on Singapore stocks are usually going upwards?
Why? Stuckist causes? Minimal volume on downmove? On the next hype,
the created demand buys and the cycle repeat? And OBV goes up further?

Volume analysis on stock index futures is okay, on commodity futures
is okay. Because Open Interest must go to zero on settlement/delivery
dates, when that delivery month futures contract is no more, cease to
exist. For futures, volume analysis is okay, very okay.




This post has been edited by csk on Jul 29 2007, 12:43 AM

hey back to this post. i do not understand why composite dollar volume is not useful

There have been incidents where customers claimed not receiving
our Order Confirmation email. In the past this was traced to our
emails ending up in the Yahoo bulk folder. What this means is that
our emails were wrongly filtered as spam. Strange why this shoud
have happen when we do not send out any spam in our history
since 1999.

Recently, there are four such incidents. We have been careful to
also email out a duplicate Order Confirmation from our emergency
Yahoo email address (only for Yahoo address customers) so that
a Yahoo-to-Yahoo non-spam email should get through. We do this
if we think there may be a problem. This don't seems to be working
and there is one very strange sitaution from a Yahoo.com.cn email
address with a Singapore address. While he could reply to our
emergency email, one which is not known, he claimed in that email
not to have received any reply. How then did he get that emergency
email address of ours?

At the same time, there were other customers with Yahoo email
address who received our emails without problems. I also have
Yahoo email adresses which I use for testing and there is no problem.

At the risk of loss of sales, but to prevent any complaint that may
affect our PayPal reputation standing, we have decided from
20-Jul-2007 to reject and refund all orders coming from a Yahoo
email address. This is clearly stated on the Order page with reason
given.

This morning an order was received and regretfully, we rejected it.
This is the first.

Many other companies do not accept orders coming from free email
providers (Yahoo, Gmail, msn, hotmail, etc.) to prevent frauds. While
I can't say yet that I may have frauds on my hand, but the handling is
very troublesome.

I am sending out registered letters with return receipt so that
neccesssary follow-up steps can be taken with proof. Two such
letters going out today.

Hopefully, these incidents at hand do not turn out to be fraud.

Hi hanhao,

I checked the program codes and I did not use the
DeleteUrlCacheEntryA function in Wininet.dll. I also just tested
for my own confirmation once more that the updating is NOT
affected by any cached copy of the earlier file dowloaded.
Analyst's DataServer always pull the fresh prices from Yahoo.
I know this is the case because I tested it many times before
this feature was released.

The reason I gave the example of Yahoo Stock Indices is to let
you know that the problem does not exist and can easily be
verified by updating Yahoo latest prices for a check, even if you
do not have any use of it.

On the intraday snapshot where updating to data files is involved,
I cannot rush the feature out as I will need to test it thoroughly.
There are many things to look out for, some of which I may not
even know until it happen. This process can take quite a while.
The priority is not to corrupt the database. So far, updating with
SGX data has been smooth since the program inception in 1999.
I don't want to risk this reliability of a good record with a rush job.

The program WILL NOT update all the stocks intraday from Yahoo
since not all stocks are traded even when not suspended. I don't
want to risk any querks from Yahoo to mess up the integrity of
the database.

So it will mean that until the database are updated with data from
SGX after the market close, any latest price updated for traded
stocks intraday will mean that these stocks will have a Last Date
that is one market day after those that are not traded. Only when
the database is updated with SGX data after market close will
all symbols have the same Last Date and the SGX data will
replace those from Yahoo.

There will not be a prompt to ask users, as you suggested in an
earlier posting, if they want to download intraday snapshot from
Yahoo when the market is not closed and therefore data is not
available yet. Instead, users must intentionally do it and therefore
be aware of what they are doing. I have to cater the program to
users of all levels of general competency.

The tech support database will not be updated with Yahoo stock
data intraday to ensure database integrity. It has to be a clean
and reliable set.

that's great.

but i have no great need for the indices. what i need is the individual price of the all stocks and perform intraday scans. hope you can get this up soon! i am currently using MLD for intraday and hope to use your prog instead. hope you are progressing well

Hi hanhao,

Yes, I do know how to do this. Analyst's DataServer does not
download the page but download the data directly returned by
Yahoo. If I remember correctly, the data go straight to memory
first. Program then handles the data in memory before updating
to files.

I am aware of the DeleteUrlCacheEntryA function in Wininet.dll.
But I am not sure whether I inplemented it. I recall that I had
some problems with some computers. So far there has not been
problem of the program using the earlier downloaded files in the
Temporary Internet Files directory.

You can update the Yahoo Stock Indices "latest prices" during
market hours and you wil know what I mean. They are updated
with the latest data available from Yahoo. So the mentioned
potential problem is not affecting.



This post has been edited by csk on Jul 20 2007, 02:47 PM

csk, i dunno if you already know this or not, but there's a great way of getting data from yahoo without downloading the page

instructions are here
http://www.gummy-stuff.org/Yahoo-data.htm


also, you might want to clear your cache before downloading from URL to prevent the older data from overwritting new data using DeleteUrlCacheEntry API

traditionally, vwap is used to benchmark big trades.

ie i have 4% worth of singtel to offload. i cannot just sell and i do not have the expertise to sell it. because i do not want to have significant market impact.

so i turned to my broker for help. one of the way he can help me is to buy my shares. not at market price but at vwap-x price.

nowadays vwap is also being included in many algorithm trading packages. vwap is used to determine the shortfall and thus the aggressiveness of one's trading.

VWAP (Volume Weighted Average Price)

Sum(Price * Volume)/Total Volume

From your example:

1) Sum (Price * Volume)
0.540 * 1000 = 540
0.545 * 433000 = 235985
0.550 * 843000 = 463650

Sum = 700175

2) Total Volume
1000 + 433000 + 843000 = 1277000

VWAP = 0.5483 (700175/1277000)

The SGX source has the numbers to calculate EOD VWAP but the
problem is as what I have explained. If only Equis had used the
entire CompuTrac format spec (instead if just the 5 and 7 fields)
then we would not have the problem today.

The CompuTrac format is more flexible but the TA program must
be able to plot the data in whatever way possible instead of just
the OHLCV in the conventional way that MetaStock is designed.

Dollar Volume can go into a field and designated as Dollar volume
and TA software must read the CompuTrac DOP file to recognise
the field as Dollar Volume (edited to correct from Trade Volume)
and therefore not assumed it as a standard MetaStock OHLCVI
fields and therefore avoid wrong calculation usage.

But, ALAS, this is not to be with the MetaStock format and the world
no longer read the CompuTrac format as itwas designed but
instead read CompuTrac format treating it as the old MetaStock
format with rigid field structure.

Cannot implement this as I foresee a lot of problem. Extra 42 meg
and nearly twice as long updating time. VWAP has to be calculated
for every stocks - processing and updating time.

Have to understand what is VWAP used for? For intraday knowledge
of ongoing intraday VWAP to determine whether the price is cheap
or expensive? Or for EOD charting which will only be a dot or line
chart. What is the advantage of this over the OHLC chart?



This post has been edited by csk on Jul 19 2007, 09:15 AM

actually talking about this
sorry if i named it wrongly
(Posted Image)

This post has been edited by hanhao on Jul 18 2007, 12:13 PM

But then I still don't dont what is the formual for volume weighted price.
same here.
are you by any chance talking about vwap ?


volume * typical price
where typical price = (HIGH + LOW + CLOSE)/3
metastock can do this as an indicator

This post has been edited by Undead on Jul 18 2007, 07:23 AM

If meaning to update the Total Dollar Volume everyday for every
stocks, it means another complete directory of stocks database
each file having the Total Dollar Volume in ithe OHLC field.

Although this can be done I am not sure if it feasible and acceptable.
The database will increase by approx 42 meg for the 790 stock
symbols. Daily updating will take nearly twice as long.

Most TA software, especially MetaStock cannot load two symbols
into one chart and do any calculation or manipulation of data.

I think not practical. Very likely may have more trouble from users
than bargain for.

But then I still don't dont what is the formual for volume weighted price.
So if it is for any numbers calculated from the OHLCV fileds then this
can be very easily done by TA software as an indicators.

store this as a new folder "C:\TAR-Data\SGX\Stocks_VolWeigh" with exactly the same name

There may be different description and formula as it is common
for many to throw in their own interpretation that, honestly, now
I don't know what this actually is.

Is there a good, reliable source for the explanation?

The SGX provides Total Dollar Value traded for the stocks but
where to store this if you are referring to keeping historical data.
The MetaStock format does not cater to custom fields but the
CompuTrac format (hich MetaStock is derived from) does cater
for custom field. I cannot insert this or other calculated fields
into individual stocks data file as it will mean the same problem
in my 15Jul2007 posting all over again.




This post has been edited by csk on Jul 17 2007, 06:38 PM

what about daily volume weighted price ? is it possible to retrive this data from SGX? this is another useful information

This is funny.
But CSK has highlighted an important point.

Now let us expand the example further. The 3 component stock has
changed over time. It became B, I, G. Then L, I, E. Then A, N, D.
Then N, O, W.

Go further the Index was revamped to better reflect the market.
Component stocks was increased to 4 (from 3). They are now
S, H, I, T.

This is why when someone commented about the history of Sesdaq.
I got to ask, is ADX then the same as ADX now ?

Applying TA on stock index is unlike TA on FX.

Intraday snapshot is not something new in Analyst's DataServer.
It is already implemented although not for the SGX stocks.

During market hours if you update the Yahoo Worldwide Indices,
you will get the latest price for the current day updated into the
MetaStock database. Updating multiple times a day will update the
current day prices with the latest available price info from Yahoo.

There will need to be more stringent check for SGX stocks as
the possibility of unexpecetd querks from Yahoo may introduce
errors.

One example is that Yahoo used to provide previous day's prices
for current day if current day is a holiday. Or they (through their
data vendors) provide erroneous data for holidays, like "0" or
previous day's Close. This will of course introduce updating errors
which can potentially be difficult or cumbersome to correct.

Right now, Yahoo do not have this Mr. A B Normal behaviour but
there is never any quarantee that this won't come back in the
future.

What potential problem can this cuase? I am working on auto
historical adjustment of stocks for corporate action. If an error is
introduced and users only found out much later then they have to
delete the error data or re-update. This will affect the Hist Adjust
already done therefore losing synchronisation. A re-update of
daily data for the affected date will update unadjusted data for
that day so Hist Adjust have to re redo.



This post has been edited by csk on Jul 15 2007, 03:44 PM

Yes, there are limitations in using Trade Volume vs Dollar Volume.
Trade Volume is generally known as simply Volume so I will just
use this term.

Back in 1999 and 2000, I updated individual Dollar Volume into
the stocks OpenInterest (OI) field but there were problems.

Some users were using technical indicators that uses OI in their
calculation. Finding data in the OI field, the indicators use them.
Naturally, the indicator values became wrong.

Then there are questions on why stocks has OI. Good questions
actually because OI only applies to futures.

So, in the end, I took them (Dollar Volume) out.


I have received a common question a number of times:
"How come SGX Indices do not have Volume?"

My reply, which is correct for the situation, is that the sources
(SGX and Yahoo) do not provide them. Well, Yahoo actually
provide for STI only.

Using Volume in a composite way (meaning in/with Indices)
has problems associated with it. There are those immediately can
come to think of, there are also those cannot come to think of.

Take for example a simple index has 3 component stocks, A, B, C.

A has gone through many stock split, bonus issue, new issue via
placement, warrant exercise, etc.

B, on the under hand, has gone through numerous share conso
for reasons best known to themselves and their accountants.

C has remained status quo since day one.

Is the Index "historical" volume affected? If yes, what use is volume
studies on Index?

Now let us expand the example further. The 3 component stock has
changed over time. It became B, I, G. Then L, I, E. Then A, N, D.
Then N, O, W.

Go further the Index was revamped to better reflect the market.
Component stocks was increased to 4 (from 3). They are now
S, H, I, T.

All the while, during these changes, the Index value was carefully
re-weighted to wipe out any natural Index price gaps due to the
change in component stocks.

Still believe in this type of Index and its Volume studies?

well i do agree with the volume might be caused by penny stocks. this volume really throws the readings off!

what i would suggest is instead of volume, we have money instead.

volume * typical price

where typical price = (HIGH + LOW + CLOSE)/3

wont that be a great broad market indicator?

yes that would be great. i hope i eagerly looking forward to seeing this feature!

Hi hanhao,

Intraday snapshot is something I have thought of. In fact it is still
on my own wishlist for the product. If you scroll to a post I made
in this thread on 28Mar2007 5:41 pm, you will know what I mean.

One thing I cannot do is to parse data from the SGX LiveQuotes
pages. This is against their Terms of Usage of their website and
I do not want to risk them cancelling the permissions they have
given for Analyst's DataServer's uses. I don't want to be the source
of unneccesary load on their LiveQuotes pages.

Do note that the prices from LiveQuotes pages are not that "live".
I do notice that broker's transaction Bid and Ask can be different
from the LiveQuotes pages even when I do a refresh.

What I can do is to get the intraday snapshot from Yahoo with the
20 mins delay. As you have noticed, the symbols are just the
addition of the ".SI" suffix. This was not so previously before they
changed to the present vendors. There is always a possibility that
should they change vendors again that this may change, or maybe
not.

I will see what I can do on this.

I have written quite a number of codes snippets of things that I can
do with Yahoo. It is unfortunate that Yahoo does not provide
fundamental datas for SGX stocks the way they do for US stocks.
Otherwise that will be a wealth of info possible for Analyst's
DataServer without the prohibitive high charges like elsewhere.

i just realised that SGX didnt not provide opening prices. this is how i think it should be done

- onUpdate, download current day's EOD data from SGX
- if current day's EOD data not available, prompt user if he would like to download an intraday snapshot,
- if yes, prompt where should the intraday snapshot be downloaded ( or make it default to SGX's page)


SGX intraday snapshot
- parse data from their webby.
- to solve the problem of not having open data, grab it from yahoo instead. open data would not change
advantage of SGX intraday snapshot : updated realtime
disadvantage : no open field. the open field would have to be taken from yahoo which will take longer

yahoo intraday snapshot
- parse data from their webby, each and every symbol. add ".SI" to the SGX symbol to make it a yahoo symbol
advantage : all data is there. including the open data
disadvantage : 20 mins delay

hey if you wish i could give you my C++ MFC class for parasing data from their webby :D

This post has been edited by hanhao on Jul 13 2007, 03:06 PM

hey CSK, thanks for the data program

but one thing that is missing is the intraday snapshot. i use it very frequently to catch intraday breakouts. dont wan to miss the boat. could you please implement an intraday snapshot and put it to EOD data? here's the pseudocode

- grab data from SGX
- parse the data from http://stquote.sgx.com/live/st/STStock.asp...t;alphabert>
- append this data

also this allows users not to have to wait till 630 for EOD historical data. This is what MLDownloader has been doing with yahoo source

just 1 problem i foresee is that SGX says

to workaround that problem make the URL changeable and flexible

also you could include yahoo data source too incase SGX dies. too bad yahoo data is 20 mins delay

This post has been edited by hanhao on Jul 13 2007, 12:19 PM

Hi teachme,

They are in the email circular:

1) Auto-skip-over-holiday when updating.
In version 2.1.n, holidays were auto skipped over when updating.
When re-arranging the flow of the program codes in version 3.0.0
for one-click updating, this auto-skip-over-holiday was inadvertently
broken. This was found out after the Vesak Day holiday in May.
Version 3.0.3 now re-implement the auto-skip-over-holiday updating.


2) A new Export to CSV dialog.
You can now specify the Date Format, whether or not to have a
Header Row and whetehr to use the StockName or Symbol for the
name of the CSv files. This offer flexibility for users who need to use
CSV data.


3) Alternate Hist Adjust window for WinXP Tablet OS.
The WinXP Tablet OS has some issues with Microsoft's comctl32.ocx
control. The Toolbar and ListView Checkboxes created by this control
cause WinXP Tablet OS to crash the program.

As this is a compatibility issue between Microsoft's own products,
there is nothing I can do except to implement another Hist Adjust
window that do not have the Toolbar and ListView Checkboxes.
If Analyst's DataServer 3.0.3 identify a WinXP Tablet OS then it
shows the alternate Hist Adjust window.

All other Windows users will see the normal Hist Adjust windows with
Toolbar and ListView Checkboxes.


4) There were a few incidents where the SGX source file downloaded
was incomplete. Usually this is due to Internet connectivity issue that
covers also the stability of the SGX website at the time of downloading.

The incomplete SGX source file caused Analyst's DataServer not able
to parse the source file properly and trigger an error during the next
program launch. This is now fixed. If the SGX source is incomplete
and SGX encounter a parsing problem then it will clear all the ListViews
and you will see them empty. The program will still launch and run
properly.

Analyst's DataServer will also stop updating an imcomplete file if
detected and will therefore not change the Last Date. The next time
Analyst's DataServer is launched, it will detect the previous date as
the Last Date.


5) New Ledger Background Color for Windows Vista.
The standard row height of the ListViews in Windows Vista is changed
from previous Windows. This makes the previously provided Ledger
Background Color to be out-of-alignment in Windows Vista. Now
version 3.0.3 has a different set of Ledger Background Color specially
for Vista. Analyst's DataServer will check for the Vista OS and if
running, it will apply the Vista set of Ledger Background Color.


6) Resolving the last of the Windows Vista ActiveX registration bugs.
Analyst's DataServer version 3.0 was re-written to be fully Vista UAC
compliance. This means that Vista users must run Vista with the
default UAC behaviour. This is Microsoft recomendation and any
deviation will result in user encountering problem in Vista.

In Vista, even when you have Windows login Administrator privilege,
your privilege will still drop down to the lower Standard User privilege
when running programs. This is a major change from all previous
versions of Windows. Hand-in-hand with this change is also the way
Vista now handle ActiveX control registraton in the Windows Registry.

Disabling UAC and running programs permanently as Administrator
privilege without dropping down to Standard User privilege will invoke
339 and 429 errors pertaining to ActiveX registration. Some of these
cannot be resolved. This is not a problem with any program but rather
with the requirement for UAC. This ActiveX registration bug in Vista is
neither acknowledged by Microsoft nor is there any signs that it will be
changed in a future Service Pack..Since Analyst's DataServer is
designed to Microsoft guideline then it must be run with standard Vista
behaviour which means UAC must be turned ON. UAC is ON by default.
Note this is NOT a problem or bug in Analyst's DataServer.

Change in MetaLib.dll installation location.

The MetaLib.dll ActiveX component is now installed into the Windows
system directory. This is usually C:\Windows\System32. Previously it
was installed into the program directory but on some (not all) Vista,
this throw a 429 error. Therefore this change in install locaton.

To prevent compatibility issue, there are now extensive checking
codes for MetaLib.dll on program launch. Incompatibility problem can
be caused by installing another program that install an incompatible
version of MetaLib.dll. For Win9x/ME/NT/XP/2003, Analyst's
DataServer will try to correct the problem if any incompatibility is
detected during program launch. It does this by copying a compatible
version into the C:\Windows\System32 location and registering it in the
Windows Registry. However, in Vista, due to the C:\Windows\System32
location being protected by UAC and the Analyst's DataServer cannot
do ActiveX registration, it cannot do an auto repair. It will therefore
pop up a dialog box to re-install the program.

Analyst's DataServer 3.0.3 was released about an hour ago.

To workaround the Windows Vista ActiveX registration bugs,
I tried to implement Registry-Free COM (link) but it doesn't work
in Windows Vista. It is supposed to but reality is different from
theory. It works only in WinXP SP2 (and Win2003). Understandably,
it will not work in older versions of Windows because the technique
was not built into them.

Since my purpose of trying Registry-Free COM is to try to
overcome Vista ActiveX registration bugs and it does not work,
I have to abandon it.

Windows Vista is a pain-in-the-@#% OS. With all these bugs and
UAC restriction, I will stay with WinXP for quite a long time more.
I have no choice. TradeStation 2000i cannot be installed in Vista.
If someone were manage to force it in, I know it still won't work
because of UAC. So are Microsoft's older developer langauges
from the still popular Visual Studio 6 right up to Visual Studio 2003.

So WinXP, come stay with me.

Hi csk,

By luck, I discovered that DataServer ver3.0.3 has been released today. Maybe I am among the first to use it.

May I know what are the new features?

The Ledger Background Colors for Windows Vista was finished over
the weekend. Analyst's DataServer now checks for the Windows
version and if it is Vista, it will then use the appropriate Vista set
of Ledger Backgrounds.

One user who recently started using WinXP Tablet Edition experiences
a conflict problem between Microsoft's comctl32.ocx and the Tablet OS.
The Hist Adjust window's Toolbar and Checkboxes User Interfaces
created via comctl32.ocx clash with the Tablet OS. Not sure why
this should even happen but since it is a Microsoft's problem there is
nothing I can do to comtl32.ocx. comctl32.ocx makes calls to
comctl32.dll which is now a Windows system protected file and from
WinXP onwards this file is not re-distributable.

To enable the user to still use the program in his WinXP Tablet PC,
I am adding in program codes to check for the Tablet OS which will
use a different Hist Adjust window to work around the Microsoft's
conflict problem.

Analyst's DataServer ver 3.0.3 will be released once this is done in
the next few days.




This post has been edited by csk on Jul 6 2007, 02:01 AM

From feedback, a lot of users like ver 3 for its improved operational
fucntionality.

A minor upgrade ver 3.0.3 will be released soon. Aside from some
codes to fix and improve the program, this release improve the
Export to CSV feature. Now, users can specify to use the stockname
or the symbol for the exported CSV filename. Three date formats
are possible - yyyymmdd, mm/dd/yyyy and dd/mm/yyyy. The is
an option to either have or not to have a header row (for the fields)
in the exported CSV files.

While MetaStock and other users do not need these features, there
are users who want to have CSV files to use with other programs
that take only CSV files and do not have the flexibility to recognise
different date format, skip over the header row or read long file
names.

(Posted Image)


SGX recently changed their webpage for AGM Schedules. The link
to this webpage in the program menu is updated to reflect this change.

In ver 2, holidays were automatically skipped during updating. The
program flow (not the actual updating codes) in ver 3 were changed
to enable the improved one-click updating. However, this unintentionally
broke the auto-skip-over-holiday functionality. This was found out after
the Vesak Day holiday. Ver 3.0.3 re-instates this auto-kip-over-holiday
funtionality.

Ver 3.0.3 release is now pending the inclusion of Ledger Background
Colors for Windows Vista. Should be quite soon. Definitely way before
the National Day holiday to have ample time for users to upgrade.

(Posted Image)

[Deleted] Problem solved

This post has been edited by teachme on Jun 22 2007, 09:55 PM

Hit a similar show-stopper with the second US web-hosting provider.
So after a weekend of frustration, I am now back to my original
web-hosting provider yesterday. Had to re-create quite a fair bit of
things. The website was progressively re-built yesterday and this
morning.

Everything is back to normal now.




This post has been edited by csk on May 29 2007, 12:22 PM

Well, there was a feature that was not available at the new web-hosting
provider and this was a show stopper. After some emails to and back,
it was concluded that the package was not suitable and I could cancel
and make a refund request. So I called them (USA) this morning and
confirmed all that was said in the email and process my cancellation
online while on the phone to ensure everything got along smoothly.

I was in a fix. I had to get another web-hosting company immediately
or revert back to my present Singapore one. I called them. No luck,
they are not working on weekends. I need to ask them something
before I would continue. Sadly, last month when I gave termination
notice, they said that the advantage of hosting with them is that they
are only a phone call away. Yet they were not there.

Luckily, I had done some homework before I made that "wrong"
choice. So I called the other US web-hosting company to ask and
make sure this time that the feature is available. Once everything
is confirmed, I signed up online. They are the registrar of my US
domian names so I have dealings with them.

Now, it is waiting time for the Domian Name Server for my new
hosting to propogate the Internet before I can get back access of
my domiain. It look very likely now that my website will only be
accessible the earliest tomorrow morning. Right now it is not
accessible. I don't have email access for my domian too.

Right now, the graphics I posted in this forum are all blank because
the links are not working (can't find the domian, therefore the files).
Should be back to normal tomorrow when I can upload all the files
onto the website with the new web-hosting provider.



This post has been edited by csk on May 27 2007, 08:29 PM

OMG~ :o for some reasons, I've removed the Uninstall option of v2.1.3 from the Control Panel. There is no alternate way for me to uninstall v2.1.3 (as there is no such uninstall exe file in DataServer program directory)

Argh... my itchy hand :angry: headache... :(

On one hand, I can't install new version, on the other hand I can't uninstall old version. Now I'm stuck...

This is the result of itchy hand :angry: Finally my installation problem on WinMe was solved. What have I done?

After some tweak in mind, instead of searching for uninstall manager, tools etc, I did a search on restore registry particularly for WinMe OS. And I found the below DOS command:

At the C: prompt, type scanreg /restore, and then press ENTER
(full details can be found in this article at MS site)

But warn you! For those who also have face corrupted registry problem similarly like what I'm facing, you might have to reinstall many programs that you've presently installed on your pc/laptop. This is an alternative to the last resort - i.e. re-format the harddisk <_<

And finally, both v2.1.5 & v3.0 of DataServer were installed smoothly (thou' there is no need to install v2.1.5 actually.. just wanted to test out).

It's a pity I still unable to resolve the installation problem in my laptop, becos the DOS command is not recognised in WinXP, and since there was no previous Restore Point that has been created, I'm unable to restore back at a very much earlier state.

Nevertheless, it's still an improvement :rolleyes:

This post has been edited by culion on May 21 2007, 05:52 PM

The entire Windows software library in the world has been turned
upside down with the release of Windows Vista.

Fortunately, Analyst's DataServer will run on Windows Vista. To
make its installation and first run seemless, I have been doing quite
a bit of install/run tests of Analyst's DataServer on Wndows Vista
for the best install and first run combination. I think I have found it.

To see a serious bug that is affecting a lot of existing software, just
do a search on Yahoo or Google with the ALL the following
keywords - Vista + Error + 339 + OCX - and see all the problems
kicked up by a serious bug in Windows Vista. The correct way
of rectifying the error 339 will not work. Microsoft is still not
acknowledging this bug and their engineers are still giving the
wrong instruction (previously correct) to register the OCXs. Yes,
registering OCXs is the correct remedy but ONLY on all previous
of Windows!!! It won't work in Vista!!! So even Microsoft engieers
are stumped.

Now, not only are developers over the world not prepared for
the nasty surprises sprung by Windows Vista, but Microsoft
themselves are not spared too.

Microsoft Visual Studio 2005 is the latest released program
development product from Microsoft. Yet, there are issues running
it in Windows Vista. See this Microsoft website link for the extent
of the problem. I just counted there are 48 issues in the list.

Visual Studio 2005 on Windows Vista Issue List
http://msdn2.microsoft.com/en-us/vstudio/aa964140.aspx


Note: Visual Studio 2002 and Visual Studio 2003 are no longer
supported by Microsoft because they won't run on Windows Vista.


Consideration if you are itchy for Windows Vista:
On the same hardware, Windows XP runs so much faster than
Windows Vista. If your hardware specs meets or exceeds Windows
Vista requirement, running Windows Vista will give you just a so-so
speed experience. If you install Windows XP on that computer, it
will take you to the moon in 1 sec. That is how much diffferent the
user experience is, with only a bit of exaggeration.





This post has been edited by csk on May 20 2007, 02:05 PM

HI Lizard,

SGX-DT was a natural since it is the other half of SGX. However,
its availability the following day at around 2:00pm make it
useless for timely analysis. This has not changed for many years.

At 2:00 pm:
1) MSCI Taiwan regular T session has already closed at 1:50pm.
2) Nikkei225 regular T session is about to close at 2:30pm
3) Active part of Mini-JGB trading is about to be over (Tokyo)

For this reason, SGX-DT was not included.

Hi CSK,

Any plans to include SGX Futures into the Dataserver? 2 thumbs up for Analyst's Dataserver :)

This post has been edited by Lizard on May 18 2007, 10:31 PM

Why I use the Budddhism analogy of Cause and Effect.

One important thing I learned in programming is that I must think,
do and act logically. If the result (actual effect) of the program
codes is not working the way I intend it to (desired effect) then
the problem is in the codes (cause).

I cannot blame the programming language, I cannot blame the
operating system. They always will take my instructions (through
the program codes) and execute accordingly. They do not know
what I want (desired effect) but they will always produce the
result my codes instruct (actual effect).


Here is what I understand of culion's experience. I am guessing
based on screen capture and culion's observation:

1) Start Analyst's DataServer
2) MSI do a repair of the installation
3) TarADS.exe.local re-installed
4) Windows rebooted?
5) Delete TarADS.exe.local file (must do 'else program won't run)
6) Start Analyst's DataServer
7) Program run successfully (becos TarADS.exe.local deleted in 5)
8) End program

At this point, TarADS.exe.local is no longer present since already
deleted in step 5.

The next time:

a. Start Analyst's DataServer
b. Cannot start because TarADS.exe.local is present ***
Therefore:
c. MSI do a repair of the installation
d. Tar.exe.local re-installed
e. Windows rebooted?
f. Delete TarADS.exe.local file (must do 'else program won't run)
g. Start Analyst's DataServer
h. Program run successfully (becos TarADS.exe.local deleted in f)
j. End program

At this point, TarADS.exe.local is no longer present since already
deleted in step f.


The next time:

a. Start Analyst's DataServer
...
...
...
The cycle repeats (samsara - cycle of rebirth and death)


***:
Where did TarADS.exe.local come from? It was already deleted in
step 5 and also step f on every cycle. There is nothing in the
program code to re-create it. I know this for a fact because I wrote
the program codes. So where did it come from? Or was it not there
already at that point in time? If it is not there and it is percieved to
be the cause then why did the program fail to run properly? What
really is the Cause of this Effect?


Military TOPO analogy:

When I am lost in the jungle, I cannot blame the compass that it
has suddenly stop pointing to the North. I cannot insist that the
compass has now start pointing East. If I do, the next morning
when the sun rise, I will insist that the sun now rise from the North
and set in the South. I will remain lost in the jungle forever.



This post has been edited by csk on May 17 2007, 10:55 AM

Hi culion,

From what I understand, from your screen capture and what
you explained, MSI does a repair everytime you start Analyst's
DataServer. So on every start, MSI re-install any missing files
and this include TarADS.exe.local which you delete everytime.
MSI has a database record of what are the files in the package.

I believe because MSI does a successful repair everytime
(or it think it does), this is why it ask you to restart Windows.
Normally, MSI installation and repairing programs do not
require a restart of Windows. This is one of the thing Microsoft
designed in MSI so that Windows users don't have to reboot.

I am assuming that because of this repair, you are able to run
Analyst's DataServer (even though it still spring a repair every
time). I am assuming that becasue of this, your deleting of the
TarADS.exe.local may be only coincidental of the program
running.

I cannot say any more of the only purpose of the .local file.
I think even if you do not delete it, the program may still run
because of the MSI repair.

Buddhism analogy:
Everything has a cause and effect. What we see are usually
effects. We usually don't see the causes. And when we try to
find the causes, we tend to associate the wrong causes to the
effects. Once we associate the wrong causes, we try to correct
the wrong causes. Because the wrong causes are corrected,
its corresponding effects are corrected. But the effect that we
want to correct is still not corrected because its cause is still
not correctly identified.

When we want to stop our existence in samara (the repeating
cycle of rebirth and death) and attain enlightenment, we must
immediately learn to stop creating the causes for cyclical
existence and immediately start creating the causes to attain
enlightment.

Causes already created and accumulated in the past translate
to our karma. Our karma goes with us. It is like a store house.
Whatever causes we have created are stored there and their
effects will one day, when the conditions are right, ripen. The
effects once ripen will take their full course. Nothing can stop
them.



This post has been edited by csk on May 17 2007, 09:16 AM

Hi CSK,

Don't get me wrong. As I've mentioned previously, I'm not trying to find flaws in DataServer, nor am I trying to blame anyone or not to believe any suggestion. And I already mentioned that I might have already screw up the registry myself.

I'm only showing the "real situation" that I'm facing. And the program will only works (so far as of now, no other ways) after the local file is deleted. I'm not trying to point that this local file cause the problem, I'm only trying to show that this is so far the only solution that will make DataServer works.

Once again, what I've been posting down here is only trying to share the experience/encounter I face (and perhaps a gentle warning to others not to anyhow mess with windows registry). Who knows someone else might face the same problem? I'm happy enough that I finally make the DataServer works even though I have to keep repeating the two simple steps.

Once again, like I said ealier, I give a thumbs Up for your program. No worries...

Hi culion,

I can see that MSI is doing a repair when you run Analyst's DataServer.
I explained earlier that this may be the case but I was not sure. As your
screenshot showes, MSI re-created the file or I should use the correct
term, re-install the files because in its database (MSI has a database)
there is this file.

As I have said, the existence of TarADS.exe.local is for the prevention
of DLL Hell and to enable side-by-side installation of DLL. If you do not
believe me and more importantly also do not believe Microsoft (the
creator of Windows who created this behaviour in Windows) and insist
that TarADS.exe.local is causing the problem for you then there is
nothing I can say anymore except to parrot Microsoft once again:

The End of DLL Hell - Microsoft MSDN
http://msdn2.microsoft.com/en-us/library/ms811694.aspx

Once again, the article is long. Do a search for .local to go straight
to the part that explains the purpose of the file directly.

This is the reference to what Microsoft explain about something
they designed for Microsoft Windows right at Microsoft MSDN
website. I don't know how more to explain what Microsoft say.
When it comes to Microsoft Windows, Microsoft is the authority.
When Microsoft says if you program for Microsoft Windows and
you want to prevent DLL Hell and you want to do side-by-side
installtion of DLL then you include this file. In the End of DLL Hell
article they even say the .local file is an empty file. I follow the
authority instruction to include the file exactly for the purpose.
There is no reason for me to suspect that he file will screw up
my own program.

I have to leave you to decide whether you want to believe or
not to believe. I have tried and failed. In the same vane, I think
Microsoft has tried and failed. I cannot force you to believe
something you do not want to believe.

You have a very unstable Windows system. This is the cause of
the effect. If you don't correct the cause, the effect is always
going to happen. This is the law of cause and effect.

Hi CSK,

Let me show you the real situation. 1st, I get this error again when attempt to launch DataServer (note that I did not edit the TarADS.ini file, it remain as it is in the $AppData$ folder):
(Posted Image)

Once again (as said in previous posting), I delete the TarADS.exe.Local file and launch the program as shown. Attached is the captured screen showing the Recycle Bin as well as Program Folder & what happen after clicking TarADS.exe:
(Posted Image)

As you can see, TarADS.exe.Local is being re-created again. That is why I said I need to perform this two steps above in order to get the program works again after each time it is being exit.

In fact, I already suspected that I've screw up the registry after making many attempts to edit along the way when trying out new freeware/shareware. I will try out the CleanUp Utility later. Tks for the link

This post has been edited by culion on May 16 2007, 11:44 PM

I found something on Microsoft website but don't know whether
it will help to correct the MSI issue on your computer.

Description of the Windows Installer CleanUp Utility
http://support.microsoft.com/kb/290301

Do take note of the first paragraph:

Warning The Windows Installer CleanUp Utility is provided "as is"
to help resolve installation problems for programs that use Microsoft
Windows Installer. If you use this utility, you may have to reinstall
other programs. Caution is advised.

"However, a new problem encounter

When I try to relaunch the DataServer program, same old
error occur (i.e. the one prompting for wrong data directory in 1st posting).
Once again, by removing the empty TarADS.exe.local file and relaunch the
program, the problem will be temporary resolved."

You mean you get the "??" instead of "C:". I think you may want to check
your Windows. Also, I notice in your screen capture whereyou show it, you
did not use notepad as you said. Instead you use WordPad or Word because
I see the "Ruler" which Notepad does not have. Why not use Notepad to
type over it again. Notepad does not save in Unicode and defintely save in
TEXT format.

Regarding TarADS.exe.local, I have already posted a link to Microsoft MSDN
webpage. MSDN stands for Microsoft Developer Network. It is the official
source and clearly states the reason for the ".local" file. I can't do any more
explanation other than keep parroting their official word.

I don't understand what you mean by:
"I need to perform the two steps shown above once again"

Do you mean deleting the TarADS.exe.local file again everytime is one of
the two? This file once deleted is gone forever. It will not be re-created by
the program. There is no way you can repeatedly delete it. Unless MSI do
a repair every time you start it but I am not sure whether MSI will put back
the file should it perform a repair.

From what you explained you tend to delete files, I think you already have
a very messed up Windows.



This post has been edited by csk on May 16 2007, 09:17 PM

You may want to do a search on Yahoo or Google for Uninstall Cleaner or Uninstall TuneUp or Uninstall Manager or some phrase like that. Just make sure the program does not install spyware or malware onto your computer.

Use it to clean up that part of your Windows system.

I've install a couple of them but of no avail. Basically it is very similar to Add/Remove which only list those that are shown in the Control Panel. DataServer program did not appear in any of these uninstaller manager.

Now the only possible solution:
- get the v2.1.2 copy and reinstall again
- edit the Add/Remove registry string (only if I know how <_< )
- format the HDD (not worth man.. just becos of one program :( )

Regards TarADS.usn, you're right.. only left a copy at the $AppData$ directory, no more in the program directory

:blink: However, a new problem encounter :blink:

When I try to relaunch the DataServer program, same old error occur (i.e. the one prompting for wrong data directory in 1st posting). Once again, by removing the empty TarADS.exe.local file and relaunch the program, the problem will be temporary resolved.

Why I say temporary? Becos now everytime when trying to relaunch the program after exit from it, I need to perform the two steps shown above once again.

I wonder how? I wonder why? I wonder where they are? (sounds familiar??)

Tentatively have to work on this way, bobian... :unsure:

Hi culion,

You may want to do a search on Yahoo or Google for Uninstall
Cleaner or Uninstall TuneUp or Uninstall Manager or some phrase
like that. Just make sure the program does not install spyware or
malware onto your computer.

Use it to clean up that part of your Windows system.


"As for "hidden files" - understand that there isn't any left after
successful install (except that there might be a copy of .MSI file
kept in somewhere, most likely in $AppData$, Temp or Downloaded
Installation folder... not confirmed)"

The MSI is kept in "C:\WINDOWS\Downloaded Installations". You
should also see some other MSI there.


"Last, I'm not trying to find flaws on your program (I myself has
a thumbs Up too!), just sharing some installation experience &
encounter along the way"

I appreciate the feedback. This has resulted in me taking out the
clean up code because of potential problem on first run. Thanks.

IBM did a study and reported that about 30+ percent of software
failure was the result of installation problem. This is why I choose
Installshield and MSI is a must for me. Microsoft came out with MSI
for proper Windows program installation. In the past there was no
MSI and installing/uninstalling was like the "Wild Wild West", no law.

OMG~ :o for some reasons, I've removed the Uninstall option of v2.1.3 from the Control Panel. There is no alternate way for me to uninstall v2.1.3 (as there is no such uninstall exe file in DataServer program directory)

Argh... my itchy hand :angry: headache... :(

On one hand, I can't install new version, on the other hand I can't uninstall old version. Now I'm stuck...

Hi culion,

I am not sure why this happen. Once you have upgraded to 2.1.3
then MSI should not have references to 2.1.2 anymore.

MSI uses Product Code and Upgrade Code to track. Product code
is a GUID for the version of the program and is changed from
version to version while Upgrade Code is a GUID for the life-time
and remains the same for Upgrading purpose.

A same Upgrade Code with a different Product Code in the MSI
means it is an upgrade. It will then check for the Product Code
of the older version to determine upgrade qualification.

Since the 3.0.0 MSI is checking for the 2.1.2 MSI then it means
you ARE upgrading from version 2.1.2 to 3.0.0. This is MSI
behaviour. As to what happen to 2.1.3, I am not sure why it
could co-exist in your system. I am not sure too whether
installing to different directory might have resulted in this. I
am not sure.

You should try to uninstall all versions of Analyst's DataServer
you find in Control Panel - Add Remove Programs. This is the
only proper method to unsintall a program so that it removes
all references to it.

Then try install ver 3.0.0.

Hi CSK,

Regards TarADS.usn file, re-installtion according to your proper steps, "hidden files & registry edit" & metalib file

I hope I remember correctly as I've tried the following ways:

1st try - delete TarADS.usn from C:\Program Files\DataServer only
2nd try - delete TarADS.usn from $AppData$ folder only
3rd try - delete TarADS.usn from both above folders

All ended with the same error prompt. Let me check again when return back home as I remember TarADS.usn still exist in both folders after the last successful installation.

Since I've finally making the installation works, I'm only half-hearted to do a re-installation as u mentioned.

As for "hidden files" - understand that there isn't any left after successful install (except that there might be a copy of .MSI file kept in somewhere, most likely in $AppData$, Temp or Downloaded Installation folder... not confirmed)
-- somehow I do find the copies of DataServer 2.1.5.msi & DataServer 3.0.0.msi even the installation fails
-- many other programs also tends to leave such a copy of .MSI file (e.g. like Adobe Reader, Java etc..)

As for Metalib file.. it's getting too chim and more chim.. and since I'm not a program developer, I'll skipped to understand the details ;)

Last, I'm not trying to find flaws on your program (I myself has a thumbs Up too!), just sharing some installation experience & encounter along the way

Cheers!

This post has been edited by culion on May 15 2007, 05:22 PM

Hi CSK,

I have a new problem, probably a very minor one...

I'm now installing DataServer on my pc running on WinMe. The last version I upgrade is v2.1.3 and somehow this time when trying to install v2.1.5 or the latest v3.0 , in both occasion it repeatedly looking for the older v2.1.2 .MSI installation file as shown (which I've removed it previously as I thought it won't be in use):
(Posted Image)

Not sure if you've removed all previous version of DataServer as I'm unable to download v2.1.2 from your homepage.

I'm curious as why the latter version requires to look for v2.1.2 in order to proceed upgrade? Can this be skipped?

Why is there an empty TarADS.exe.local file? For the prevention
of DLL Hell, for the good and benefit of Analyst's DataServer users.

Read this Microsoft MSDN webpage. It is a long article.
Search for ".local" to go straight to the part.

http://msdn2.microsoft.com/en-us/library/ms811694.aspx

"Deleting TarADS.usn did not solve the problem."

You deleted the wrong one. You deleted the one in the $AppData$
folder instead of the one in the program folder (in earlier post:-
"Go to C:\Program Files\DataServer and delete away this file.").
Because the file is already in the $AppData$ folder, see your
screen shot, the house-cleaning could not finish successfully.


"It will auto-generate again each time I tried to register the ADS
program"

Yes but only in the $AppData$ directory now, no loner in the
program directory. This change was made to be Vista UAC-
compliant. I try to make the upgrade automatic and friendly.
Should it find the old one in the program directory it tried to
house-clean by moving it over to $AppData$ thereby not
requiring you to input the registration. This is where it failed.
The file is "read-only" and cannot be over-written. This is why
it failed.

In the first instance, you shold never have two such files in
both location. It the program find the old old, it will move it
and will not require you to input again. This is how the
program codes flow.

You can check whether I was giving you the proper instruction
ealrier. Try this - copy TarADS.usn from the $AppData$ folder
into the program folder. Copy, not move so that it exist in both
places. You will get that house-cleaning error again. Guaranteed!

I have just remove all house-cleaning codes since they can
potentially do more confusion than good. Version 3.0.1 when
ready will not have house-cleaning codes. Furthermore Vista
UAC-comliance do not allow non-Adminstrator to edit or delete
file in the "C:\Program Files" directory stucture. Even an
Adminstrator ID when running programs will automatically
drop down to non-Administrator privielge to run all progams.


"run ADS program again (which seems trying to reinstall again..."

If the installation was corrupted by later manual editing or
changes, it will trigger Microsoft Installer (MSI) to "repair" it.
This is standard MSI behaviour. MSI is preferred for Windows
installation and has become compulsory for Vista. Analyst's
DataServer use Installshield which uses MSI.


"(TarADS.usn is also a 0byte file and thus giving me an idea
to look into TarADS.exe.local)"

This is NOT a 0 byte file. See above


"1. remove TarADS.exe.Local (a 0byte file)"
"And somehow TarADS.exe.local is another file that also cause the problem"

This is an empty file with no content. It is there to prevent DLL Hell
and is recommended by Microsoft for side-by-side installation of
shared DLL. Its existent is to let Windows know to use DLL in that
directory instead of other directory. When windows launch a
program, myprogram.exe, and finds myprogram.exe.local in that
directory, DLLs in that directory will have priority. This is the only
purpose of having the EMPTY TarADS.exe.local file. Its absence
can lead to DLL Hell problem later.


"Perhaps the "Hidden Directory and hidden files" are not
removed in the uninstall routine."
"I've tried that by removing all relevant directories (even hidden
files & directories) that could be created by DataServer and even
the registry but still no avail."

Analyst's DataServer do not install any "hidden files or directories".
The COM components, except MetaLib for compatibility reason, are
installed into the $WinSys$ directory because of a Vista bug and I am
still trying to improve on that. If you have deleted some hidden
directories of hiddden files, they do not belong to Analyst's
DataServer and may later on affect other programs.


"As for why I get metalib v4.5 error msg, this really puzzled... "

The program inculde the license info for ver 5.0 and there has not
been any such problem before. Without the license info in the
program, it will not be able to update and create new files properly.
I just double-check the program codes and the license info was
provided before any first use of MetaLib so that error should never
happen.

With a Commercial license, I can legally re-distribute the DLL with
my program without requiring users to separately pay for one.



This post has been edited by csk on May 15 2007, 01:29 PM

Hi Undead,

In fact a clean install of v3.0 did not solve the problem. I've tried that by removing all relevant directories (even hidden files & directories) that could be created by DataServer and even the registry but still no avail.

As mentioned by CSK, it was the "house-cleaning" part that restrict the smooth run of the program. And somehow TarADS.exe.local is another file that also cause the problem which I miss to mention in the 1st posting.

As for why I get metalib v4.5 error msg, this really puzzled... :blink:

I've checked the metalib.dll from DataServer directory and it shows v5.0.0.1

Perhaps only programmers like CSK knows the reason. Anyway the problem has been resolved :)

Last, tks for helping too :)

Lion,

Nice work!

From your solution, a clean install of Ver 3 should solve the problem.
But it did not.
Perhaps the "Hidden Directory and hidden files" are not removed in the uninstall routine.

Many programs, unfortunately like to hide themselves in the labyrinth of Windows.
Perhaps its for software protection or whatever. But that's no excuse.

Once in a while,
I will have to reinstall Windows.

Hi CSK,

Deleting TarADS.usn did not solve the problem. It will auto-generate again each time I tried to register the ADS program and the same old error msg in my 1st posting appear.

However, I think I manage to resolve the problem by:
1. remove TarADS.exe.Local (a 0byte file)
2. run ADS program again (which seems trying to reinstall again but awhile later, straightaway prompt for username & key) and somehow it works!

Just try out my luck after frustrated over repeated installation (TarADS.usn is also a 0byte file and thus giving me an idea to look into TarADS.exe.local)

So far so good. Tks... :)

Hi culion,

The program was trying to do house clean up for a hassle free
upgrade. It was trying to move the file, TarADS.usn, from the
previous version to the $AppData$ directory. For some reason,
this could not be done successfully on your system.

Go to C:\Program Files\DataServer and delete away this file.

It does not matter what directory name you install in. The program
does not hard-code its own installed path. It will know where it is
installed so "C:\Program Files\DataServer" or "C:\DataServer" will
not be a problem.

I will do ver 3.0.1 to skip this house-cleaning, when I have finish
with one testing. I am testing on Vista to see whether putting the
COM components in a private common folder can work-around
an irritating Vista bug associated with COM registration.



This post has been edited by csk on May 15 2007, 11:03 AM

Hi CSK,

Previously, I use to uninstall the v2.1.5 before install v3.0 (actually no such need)

This morning I tried again to install v3.0 w/o removing v2.1.5
(but your default directory name is so long, so I always use C:\DataServer instead).

I've done a search for the TarADS.ini , open with Wordpad and confirmed the defaulty directory as mentioned (SGX data already extract to C:\TAR-Data):
(Posted Image)
(Posted Image)

When I run the program, this time it appear the following error (which I forget to show last nite):
(Posted Image)

:( ohh... so headache. I dun find any reason there is a difference if I install in the default directory or my specified directory, can't be this problem right?

This post has been edited by culion on May 15 2007, 09:47 AM

Yes, good catch there. I was about to post on this when I see
Undead already did.

Just for info, MetaLib 4.5 has compatibility issue with I think
ver 4.8 and later. This is why Analyst's DataServer will always
register ver 5.0 again on every program launch to ensure the
correct version is always used. Analyst's DataServer install
MetaLib 5.0 in its own private directory. The MetaLib license
for 5.0 cannot be used on 4.5. Maybe, Analyst's DataServer
privately installed version was manually replaced and this
may be why the Trial prompt popped up.

There's something weird right ?

If I'm not wrong TarADS is using Metalib 5.0
however for Lion, his version is 4.5

I suggest
(1) uninstall Metalib 4.5
(2) uninstall TarADS
(3) uninstall Database

(4) install Database ver 3 in the default directory
(5) install TarADS ver 3 in the default directory

Normally when I use use a foreign language ie chinese in naming my directory,
I may get to see ??.

Many programs will not like it.



This post has been edited by Undead on May 15 2007, 09:30 AM

Hi culion,

I think you may have tried to email me at a discontinued email
address. The support email will always work.

From the error message, the program was trying to look for
the directory, "??\Tar-Data\SGX\Indices". This is not
a proper directory path - notice the ?? where a drive
letter and colon is supposed to be. Should be something like
C:, D:, E:, etc. Did you edit the TarADS.ini file to change it?

The MetaLib prompt should never have come up because a
licensed commercial version is used. The license info is in the
program code. This is the first time it happen and I am not
sure how this come about. Maybe I will put the license info
at a much earlier stage of the program to be safe.

Analyst's DataServer does not write to the Registry. Microsoft
NOW discourage programs keeping their configuration info in
the Windows Registry. Now they encourage using a text file,
in the form of XML format, in the $AppData$ location. Using
a text file is coming back to square one to the older way of
doing configuration. I prefer the good old INI format. Much
easier to read.

MetaLib is a COM component so it must be registered in the
Windows Registry. This is doen during program installtion so
no editing of the Registry need to be done. In fact, they
should not be touched so as not to corrupt the info there,
things like GUID, CLSIDs and ProgIDs.

From what I see, the problem is likely to be the data directory
path in TarADS.ini has been edited and changed. In ver 3.0, the
TarADS.ini is in the $AppData$ location, no longer in the program
directory there. If you see one there after installing ver 3.0, it
is left there from the previous version. Ver 3.0 will delete it
when launched.

The $AppData$ location is different in different versions of
Windows. Analyst's DataServer use Win32 API call to find it.

Hi CSK,

I was unable to contact you via email (all kana bounce back) so no choice got to post my problem encounter down here. It was quite agony after hearing all the successful installation while I'm having the same repeated problem.

1st of all, I've downloaded both the installation & data file. Program installed and data is executed to self-extract in the default directory.

When I try to run the program thereafter, it always prompt this error:
(Posted Image)

And on 2nd attempt, it display the 1st error above followed with this metalib error below:
(Posted Image)

I have no choice got to remain as ver 2.1.3 so agony... <_<

Kindly help me identify the possibilities of the problem.. is it something to do with Registry Edit? Just guessing...

Tks in advance.. Lion

Hi HC,

Thanks for your compliment. Glad you like like it.



Hi Undead,

You too. Thanks.
Re your suggestion on a personal portfolio tab, what info in that tab
do you have in mind?


Windows Vista

Here in a screen shot of Analyst's DataServer 3.0 running in Vista.
The row height of the ListView table in Vista is taller than in all
previous versions of Windows. In the meantime don't use any
ledger background by choosng just white.

(Posted Image)

Hi,

I like the "warrants in different directory".
Thanks for the upgrade.

Perhaps for the next one, you can include a personal portfolio tab.

Yes, Analyst's DataServer 3.0 is great. The changes now made it more user friendly.

Thanks for your good work and keep it up.

Analyst's DataServer 3.0 was released this afternoon, about three
hours ago. I am surprised and glad to receive compliments so fast.

"You are doing a great job!
Many thanks"

"Your new version of analyst's dataserver is great !
I am impress by your effort. Keep your good job !"

I really hope users will like it better than ver 2.1.5. I know I do.

I will now (after the F1 live telecast perhaps) swap harddisk to run
Windows Vista and decide whether I should change my development
computer to Vista or to stay with WinXP to develop the next version
of Analyst's DataServer for the .NET Framework.

"There are many copies in the library, but loaned out.
Perhaps everybody is looking for the secret."

I once saw a copy, the first edition by Richard Smitten, in the
library. That edition of the book was not complete. The original
rules were omitted.


"Anyway the rules in the original book are not so straight forward."

Yes, have to re-read the Explanatory Rules many many times.
It is quite hard to grasp initially. Eventually they fall into place.
Even Smitten said there was an error in Livermore's example
but he understand it was not an error because he has Livermore's
notes. I went through Livermore's example exactly as he
explained it. I cannot find any error that Smitten claimed.


"Fortunately, it had been done before. Full codes and
explanations are easily available if you know how to look for it."

I have seen people do it for TradeStation and Wealth-Lab.
The trouble there is that they don't follow the original rules.

It is just like there are a number of people do the Turtle system
for TradeStation and Wealth-Lab but none of them follow the rules
completely. Actually, I did one for TradeStation a few years ago
that follow the rules except for excluding 4th unit pyramid
because of a limitation in Tradestation.



This post has been edited by csk on May 10 2007, 01:23 AM

Hmm ...

There are many copies in the library, but loaned out.
Perhaps everybody is looking for the secret.

Anyway the rules in the original book are not so straight forward.
(My copy is very old, I'm not sure if the updated version is any better)

Fortunately, it had been done before.
Full codes and explanations are easily available if you know how to look for it.

I plan to release Analyst's DataServer 3.0 this weekend. Hope
I don't encounter any show-stopper to delay it.

I will be changing my web-hosting provider before the month
is over. The second last week will be busy time bringing the
website over. I will be hosting it on a Windows server this time.
This will give me flexibility with Visual Studio .NET development.
I had not wanted to go the .NET path but having seen the
industry direction the last two years, I think I can no longer
avoid it.

The next major version of Analyst's DataServer is very very
likely to be a complete re-write for .NET Framework 2.0 or
.NET Framework 3.0 using Visual Studio .NET.

Presently, two programs are in full swing development, still Win32:

1) Analyst's TurtleFarm 2.0
2) Analyst's MarketKey (Jesse Livermore Secret Market Key)


Here is a shot of Analyst's DataServer 3.0:

(Posted Image)



This post has been edited by csk on May 9 2007, 06:22 PM

Thanks CSK for your explanation, understood now! :D

Hi Slammer,

"SI: This is in line with the usual practise for TA charting"

Refer to the post in this thread on 18Sep2006 on the same
subject of data adjustment for Special and Bonus dividends
(not normal dividend). Special and Bonus dividends tend to
be big. I re-link the two charts in this post. You can decide
which "usual practise for TA charting" is correct.


"So now I blur, should cash dividend be adjusted? So what's
the usual practice for charting?"

Read that post and look at these two charts and decide which
usual practise you want to follow.

Keep this example in mind:

Your share trades at $1.60. It then paid a special/bonus dividend
of $0.60, meaning you sort of cash out $0.60. The market value
of your share is now $1.00. You got the $0.60 back into your
pocket so is the price gap of $0.60 or so on the chart correct?
Or should you adjust the historical prices down by the ratio?

Edited to add: If you run a trading system on the unadjusted or
wrongly adjusted data, your system may show a loss of approx
$0.60 there. In actuality, you did not. Which one will you prefer
to use? What would you consider "usual practise for TA"?

Hope this explanation is clearer.


(Posted Image)

(Posted Image)



This post has been edited by csk on May 2 2007, 12:14 PM

Very often, I have been asked why my adjusted data are
different from the majority of other sources. Other sources
are:

1) Paid subscription data vendor
2) Real-time data vendor
3) Brokerage houses (some of them don't even adjust)
4) Yahoo Finance
5) ShareInvester.com

Yes, it is kind of difficult to avoid the questions when I am
usually the odd one out and therefore arouse the suspicion
that my adjustment are wrong.

The same have been asked about my Market Breadth data,
how come they are different from others, especially SGX.
The answer can be found in this thread in earlier postings.

And as I repeated a question to my wife's gynae years ago,
he reassured me with his answer, "I know my workmanship".
And he did.

And in similar light, as I stand against giants and so called
trusted names, I will say "I know my workmanship".

I do all my historical adjustment in accrodance to the
Center for Research in Security Prices.

In case any user may be wondering how I derived at the adjusted
prices, here is the calculation process. The CORRECT method
is how I do it. The two WRONG methods are probably what you
are seeing elsewhere.


Day before ex-date:
12Feb2007
Close: 0.690

Ex-date:
13Feb2007
Bonus dividend: $0.0875
Rights issue: 1sh for 4sh @ $0.28


CORRECT method (Bonus div and Rights issue)
using 12Apr2007 close for calculation:

1) Adjust for Bonus dividend first:

Adjust Ratio = 0.873188 [(0.690 - 0.0875)/0.690]
Adjusted Close 12Apr2007 = 0.6025 [0.873188 * 0.690]
Unadjusted 52wk high = 0.730 (18Dec2006 and 6Feb2007)
Adjusted 52wk high = 0.6374 [0.873188 * 0.730]

2) Adjust for Rights issue:

Adjust Ratio = 0.892946 [(((0.6025 * 4) + 0.28) / 5)/0.6025]
Adjusted Close 12Apr2007 = 0.538 [0.892946 * 0.6025 adj]
Adjusted 52wk high = 0.569 [0.892946 * 0.6374 adj]


WRONG method (Rights issue only)

Adjust Ratio = 0.881159 [(((0.69 * 4) + 0.28) / 5)/0.69]
Adjusted Close 12Apr2007 = 0.608 [0.881159 * 0.69]
Adjusted 52wk high = 0.643 [0.881159 * 0.730]


WRONG method (Bonus div only)

Adjust Ratio = 0.873188 [(0.690 - 0.0875)/0.690]
Adjusted Close 12Apr2007 = 0.6025 [0.873188 * 0.690]
Unadjusted 52wk high = 0.730 (18Dec2006 and 6Feb2007)
Adjusted 52wk high = 0.6374 [0.873188 * 0.730]




This post has been edited by csk on May 2 2007, 11:30 AM

Hi CSK

got the following reply from SI:

<Hi Slammer

For historical price adjustment, we do not adjust for cash dividend when there is no change to the outstanding number of shares. We adjust for stock splits, rights issues etc when the outstanding number of shares change. This is in line with the usual practise for TA charting.

For the case of Federal, we adjusted for the rights issue on the 13th of Feb but we did not adjust for the dividend issued on the 13th of Feb.

Regards.>

So now I blur, should cash dividend be adjusted? :huh: So what's the usual practice for charting?

Regards

Hi Slammer,

I always check my adjustment very thoroughly before I upload
them onto the website. For a moment there I thought I made
a mistake.

I just check my work again and I confirm that I am correct. It is
very bad for me to rejoice (also bad karma to do so) at other's
mistakes. But for this case, the others are doing it wrong again.

13th Feb 2007 is ex-date for:
1) Bonus dividend $0.0875 and
2) Rights issue of 1sh for 4sh @ $0.28

If you see or hear that other sources' 52wk high is $0.64 (instead
of $0.569) then it is becasue they only adjust for the rights issue
but not the bonus dividend. I adjust for both which is the correct
thing to do.

(Posted Image)



This post has been edited by csk on May 2 2007, 09:50 AM

Hi CSK

would like to check with you on the price of Federal, my data shows that the highest price is ~$0.57 after all the necessary adjustment, but I read from some forum like SI that the 52 weeks high of Federal is $0.64, and it seems that the forums are all showing the same $0.64 price, can you advise?

Regards

Sorry, a mistake in my reply.

"The numbers are taken from all the stocks,Trusts, REITS symbols
(excluding warrants) so it is unlikely to be skewed by the top vol
counters."

The Market Breadth numbers exclude Trust and REITs. I just checked
the program codes of Analyst's DataServer to confirm this. Only stocks
are used because they should only be based on stocks.



This post has been edited by csk on Apr 5 2007, 04:27 PM

"As TRIN is one of the indicator in the package, do you really
think that applies well to track the breadth of the market when
usually, the top 30 vol counters may well skew the reading while
the broad market is weak."

The formula for TRIN is:
TRIN = (Adv/Dec) / (AdvVol/DecVol)

The numbers are taken from all the stocks,Trusts, REITS symbols
(excluding warrants) so it is unlikely to be skewed by the top vol
counters.


"Further, I find the adv and decline no. are different from
the one in SGX."

Yes, this is correct. The reason is Analyst's DataServer is doing it
correctly whereas the SGX and other sources are NOT. Market
breadth data are calculated from the "mother" stocks only. On the
SGX, warrants are like stock options in the US. They should never
be included in the Market Breadth calculation. Analyst's DataServer
uses the correct method.

There were s0me discussions in this thread sometime around Aug 2006.



This post has been edited by csk on Apr 5 2007, 12:04 AM

Hi CSK and user of TRIN,

As TRIN is one of the indicator in the package, do you really think that applies well to track the breadth of the market when usually, the top 30 vol counters may well skew the reading while the broad market is weak.

Further, I find the adv and decline no. are different from the one in SGX.

Care to share your view?

Lastly,

I wish to construct some localised indices to track certain market sector. Anyone has any good idea to share, if perhaps, through the daily data downloaded from analyst's dataserver.

Thank You

As I was preparing the previous post, I received an email from
India:

QUOTE

Hi,
this is <snip> again. iam just enquiring whether you
have any software for intraday trades, in which iam
interested.
thanks

UNQUOTE

MY reply:

QUOTE

Hi <snip>,

Unfortunately, we do not incorporate intra-day features in our
products.

There is misconception that intra-day trading is less risky. This
is not true. From my experience, intra-day trading is actually
more risky. There are a lot of noise in intra-day data that make
analysis useless. Also, traders tend to take bigger position size
because of closer stop loss levels (based on intra-day data).
Because of the inherent data noise, intra-day stops are more
easily hit. Stress for intra-day trading is also higher.

Regards

UNQUOTE


There are a lot of interest in intra-day trading. Therefore there
will be a lot of interest in intra-day products. Is this good or
bad? Well, it really depends which side you are on. If you are
on the buy side (trader/investor), it is no good. If you are on
the sell side (brokers, data vendors, exchanges) then it is good.

Perhaps, some comments from Larry Williams may be helpful.
Although he refers to futures and commosities, but they equally
apply to stocks. Where he refers technical analysis as
mumbo-jumbo, I think he means confusing without knowing
what they are. When I attended his Inner Circle and Million
Dollors Challenge seminars, he used technical analysis. Afterall,
he also invented some technical analysis indicators.

Reference: http://ireallytrade.com/

QUOTE

I think there are 3 steps for a person new to futures and
commodities.

1) Learn the basics of how to position trade, where you see a
market that is ready for a big move so you take a position.
No day trading here; you can have a job and still do this
spending maybe 30 minutes a day. This is what most people
should be doing. It is the easiest money.

2) Swing trade. No day trading yet. Look to get in and out of
a trade in 3 to 5 days. But, I caution you, this is not where
you begin. You need to know, and have traded, the basics first.

3) Short term trading. OK, you want to day trade? This is the
final step --- but I will tell you it is the hardest, the most difficult,
and the riskiest way to trade. It is the dream of so many, to
day trade, but it is usually a nightmare. I don't think you are
ready for day trading until you have been trading for at least
a year or more.

When I teach people how to day trade I put my own hard cash
at risk, so you can really see how it works. I have made
$1,000,000 in front of students trading and gave them back
20% of the profits. It is not for most people. All the FX/Day
trading stories that you have read about (making fortunes)
are hustles. Please be very careful; use your common sense.

If you have already begun to trade...

If you are not yet making money, what do you do? Go back to
the basics! My bet is that you have not learned what moves
markets... you have looked at charts and indicators, but those
things (Technical Analysis) by and large are mumbo-jumbo,
which is why you are not making money. Just because you
have been trading does not mean you know what is really
going on.

UNQUOTE



This post has been edited by csk on Mar 30 2007, 12:05 PM

Hi Undead,

Realtime data, even through a software like quotetracker or
similar, will require:

1a) a subcription payment to a data vendor and
1b) exchange fees payable and collected via the data vendor.
or
2a) an account with a brokerage firm
2b) exchange fees absorbed by the brokerage form.

Of course you already know there is NO "free" realtime data
but I will explain anyway for the benifit of others.

In both situations, the data vendor and the brokerage firm will
already provide their own front-end software. There is no need
for a 3rd-party front-end. Therefore, companies and products
like quotetracker usually start off as OEM front-end products for
data vendors and brokerages. If it does not work out then find
a sponsor or let the product die. One example of this is
Mandarin Capital here in singapore.

In the past when data vendors did not have front-end or have
only a simple one, it was common and attractive to do this and
data vendors welcomed it. But now data vendors have improved
their front-end a lot. The practice now is you need to run their
front-end even if you are using your own, like quotetracker.
This is to protect their own front-end and business interest.

Furthermore, developing a product for realtime data requires a
developer subscription to the data feed, one for each data vendor.
These subscriptions are very expensive. They may or may not
provdide an initial free period for the purpose of developing
products but after this developer are required by the contract to
pay for their feed which developers must to maintain their own
products and to make changes if the data feed specs change,
and they do change frrequently.

---

This is why I do not know whether I should include Analyst's
QuoteMonitor and delayed data into Analyst's DataServer.
Actually, I have concern that it may lead to requests that cannot
be fulfilled. Already, I have enquiries for real-time data to be
included in the one-time fee of $88.00. Whenever, I explained
the costs, they went away.

---

The development Analyst's DataServer v3 now works exactly like
v2.1.3 but with the added new features. V3 will require a re-download
of the historical database because of the separate warrants
directory and the addition of more worldwide stock indices.

The new features are still being tested under different envicronment
and their user interface of the new features are now being added.

Let me see whether I should disable the new features first and
release it.



This post has been edited by csk on Mar 30 2007, 11:51 AM

Hi Csk,

I'm not sure why I will like to monitor delayed prices "real-time".
Perhaps you can work on it to allow other datafeeds.

Take a look at http://www.quotetracker.com/index_nn.asp

Their business model is different though,
its free but adware.

My preference is to monitor quotes in Excel
because I need to do some calculations.

---

BTW I really appreciate

Not sure when will V3 be out.
Is it possible to patch V2 1st ?

This post has been edited by Undead on Mar 30 2007, 08:48 AM

I don't know how to position this product. Right now, I am using it
to monitor the worldwide stock market. It can be set to refresh the
prices from Yahoo at intervals from 1 min to 60min.

Most of the additional indices shown here (if not all) will go into
Analyst's DataServer in version 3. I have some problem with ^AORD
and ^NZ50 early in the morning but I think I can find a solution.

I have the thought of adding another tab (after Forex) to show SGX
current stock prices (with the delayed) from Yahoo. Much like the
EOD Snapshot in Analyst's DataServer so that users can sort for
biggest volume, price gainers, price losers, % gainers, % losers.

Bascially, all that can be done in Analyst's DataServer EOD Snapshot
but with latest delayed prices. It can be run on its own, standalone,
but delivered as part of the whole package.

I don't know. Maybe if you are Analyst's DataServer user, you can
let me know.

(Posted Image)



This post has been edited by csk on Mar 28 2007, 05:45 PM

Quite a fair bit of program codes were re-written to make the software
visual theme aware. To have this feature, which is standard from WinXP
onwards, the program uses the Windows common controls and requires
a lot of Win32 API programming, subclassing and hooking. This is the
bottle-neck but it is now past. This is just one more subclassing which
I am grappling with and this should be sorted out soon.

In version 3, the EODSnapshot for the Indices, Stocks and Warrants are
now in their own tabs and any change in date in one will automatically
change that of the others. The Market Breadth data are now presented
with the Indices where previously they were not presented except during
the download/updating.

The download tab are changed. Now the Status box show a more
informative picture of what is going on during the download/updating
process.

All the warrants are now stored in their own directory, sperate from the
stocks directory.

More Yahoo world wide indices will be added.

Back to punching codes and testing and more testing. Release should
not be too far now.


(Posted Image)

(Posted Image)

It is fixed. The program now works with the changes that Yahoo has
made. A patch has just been released.

Also in this patch is the ability to update Yahoo indices while the data
is loaded and charted in MetaStock. To facilitate this, there is now an
option to update only the current day's prices without having to backfill
every time. This way user can just refresh their MetaStock chart after
update to get the latest chart with current day's price, with a 15 or
20 mins delay due to Yahoo delay data.

There was a mental block earlier when I tried to figure out all that
might have been changed by Yahoo. Playing some PlayStation 2 games
did not helped. So I relied on my trusted solution - slot Guns n' Roses'
Use Your Illusion II CD into the computer and spin it up.

The everlasting inspiration anti-war piece CIVIL WAR to inspire.
Failing this, I can always try KNOCKIN' ON HEAVEN"S DOOR. It is
important not to BREAKDOWN, and no matter what I must DON'T CRY.
Then every thing will be SO FINE.

I was about to release Analyt's TurtleFarm 1.0.2 yesterday. While
doing a final test on the executable file before I released, I notice the
Yahoo chart looked a bit empty somewhere. Then I relized it was the
x-axis missing the month and the vertical grids missing. Looking at
the status line and the data tip revealed the date to be the problem.
So I held back the 1.0.2 release which is a bug fix release. Ver 2 is
coming along with auto-scan and portfolio tesng capbility.

Well, CNY brought me that bit of more work but it helped me solved
a nagging Yahoo data problem for the last few weeks.

That is done now. The data are downloaded, parsed and then displayed.
The volume which has been missing the last two weeks can now be
updated again. Now to get the codes in Analyst's DataServer changed
and release by tomorrow. Will not be ver 3 yet.

Analyst's TurtleFarm after that with a bit more work to be done.

(Posted Image)



This post has been edited by csk on Feb 20 2007, 04:47 PM

Hi teachme,

Sorry, I did not see your questions until now when I try to see what
Yahoo charts I may have posted here to see whether an error which
I encountered last few days already existed. I am thinking that I could
not have missed such a simple mistake. Yahoo made some changes
recently and last night I had problem updating historical data from
them. So now I know the problem is due to some changes Yahoo
has made over the last few weeks.

This cause Analyst's DataServer not able to update historical data
from Yahoo since last night. Volume for some Yahoo indices were
also not updated since earlier this month and I was wondering why.

Also, Analyst's TurtleFarm is showing error dates and not able to
plot weekly and longer periodicity chart (compressed using dates)
when the source is Yahoo direct download. There is no problem if
the source is MetaStock format data.

I am making changes to Analyst's DataServer and Analyst's
TurtleFarm to work with Yahoo's changes.


"...where can I go to download a recent database?"
The link you received in the order email or subsequent circulars
(as and when any) will be the same link to download.

"Do you have a time estimate as to when your Analyst
DataServer ver3 will be released?"
A while more. Quite a fair bit of codes have to be reworked and
tested to work with visual themes and this requires Win32 API
programming and subclassing. I might drop the visual theme stuff
first to cut down time. I will think about this.

"Do you have plans to add support for other foreign exchanges?"
To do the way I do with SGX data for other exchanges will require
a team of database people conversant with foreign languages to
monitor and make neccessary corporate adjustment. Not feasible
due to the very high costs, low returns. The alternative is to use
Yahoo but it will surely miss some corporate actions. Adjusting the
unadjusted volume for split but not dividend can be done but quite
inefficient. There are tentative plans but not sure yet how it will take.

"I am also using <snip> but it is inferior to Analyst Dataserver
for SGX data."
Any software that uses Yahoo source for SGX data will have problem.
Yahoo historical data for SGX are in 2 decimals while quite a lot of
SGX stocks trade in 3 decimals - minimium tick is 0.005 but Yahoo
historical data is 0.01. Yahoo does not account for all corporate
action. Notably missing is bonus issue.


This post has been edited by csk on Feb 20 2007, 12:33 PM

Dear csk,

Thank you for taking the time to reply.

Now that some of my stock symbols are gone due to accidental deletion, where can I go to download a recent database? The worst case is to use a really old database and redownload all the data again. This will take ages, but can be done.

Do you have a time estimate as to when your Analyst DataServer ver3 will be released? I am waiting in eager anticipation.

Do you have plans to add support for other foreign exchanges? I am also using MLDownloader but it is inferior to Analyst Dataserver for SGX data.

This post has been edited by teachme on Feb 3 2007, 02:43 PM

Hi teachme,

Regarding my reply:

"If "active" symbols are flagged as having expired then they are likely
to be in used by MetaStock during the updating process and therefore
locked and blocked from being updated. "

Actually this is not phrased correctly. In some situation, some individual files
are still locked and blocked from being updated. In normal usage, files in used
by MetaStock can still be updated but no new files (IPO, new listings) can be
added or existing files deleted (expired symbols).

In ver 3, Yahoo Indices will be updated even when MetaStock has those
data loaded since no files are added or deleted in the Yahoo directory.

When come to analyzing stock indices, I always prefer the Yahoo! source
indices that Analyst's DataServer maintains. I always update every evening
so I don't have problem of today missing the previous day data.

What is this problem? It is like this. Right now it is 31-Jan-2007 12:43pm.
The data for yesterday (30-Jan-2007) is no longer the latest price and
Yahoo has not updated into their historical database yet. They will do it only
later this afternoon or tonight.

If you are like me updating every evening then you can analyze with current
day prices by just doing the "Update Asian Stock Indices from Yahoo!". The
current day prices will be updated as of the time of update with a 20 min
delay due to the delayed data source of Yahoo. You can update latest prices
any time of the day, even during the day's trading hours. See chart below.

In ver 3, there is a choice to update "Latest Prices only" to cut down
updating time when the historical backfill is not required.

(Posted Image)

(Posted Image)




This post has been edited by csk on Jan 31 2007, 12:59 PM

Hi kingpin,

I have added checkboxes to select and update multiple symbols. Here is
a screen capture of the checkboxes in version 3.

(Posted Image)


I am also thinking of how I can have the Hist Adjust done automatically
during the daily updating process. I am thinking of synchronization issue.

Version 3 can update Yahoo Asian stock Indices, FX rate, previosu day OHL
of SGX Indices and auto delete of expired warrants and stocks (optional).



This post has been edited by csk on Jan 31 2007, 10:46 AM

Hi teachme,


"Also, there was once when I clicked on "Deleted Expired Symbols" and
some stocks which have not expired were shown. The stocks were
deleted anyway and I could not recover them later."

I will explain how symbols are determined to have expired. SGX Indices
refer to SGX source and not Yahoo source.

Updating Process
During the daily updating pricess, the SGX Indices are the last to be
updated. This ensures that all other "active" symbols will already be
updated by the time SGX Indices are being updated.

Expired symbols will not have data in the source and therefore cannot
be updated.

Determining Last Update Date
To determine the Last Update Date, Analyst's DataServer cycles
through the SGX Indices to get their Last Date. Since they are always
updated last, this is used for the Last Update Date.

All the SGX symbols' Last Date are checked against this Last Update Date.
If any is found to have a Last Date earlier than the Last Update Date,
then they are flagged as expired symbols.

If "active" symbols are flagged as having expired then they are likely
to be in used by MetaStock during the updating process and therefore
locked and blocked from being updated.

I programmed Analyst's DataServer to check the directories during launch
and if they are in used by MetaStock, to flash such a message and abort
the launch. I do notice that when Analyst's DataServer is already running,
MetaStock can be launched and the data directories then become locked.
In version 3, I have move this check to the "Download" button.


Additional Info
With regard to locked directories, MetaStock has an irritating bug and
Equis seem not doing anything solve it except to tell their users how to
manually solve it every time it occur. MetaStock is known to somtimes
fail to unlock data directories when it exit. See this link:

http://www.equis.com/Customer/Support/KB/F...x?c=7&p=&fc=207

I have a few users reporting this problem thinking it is an issue with
Analyst's DataServer and I have to tell them how to solve it directly
and also refer them to the Equis link for information that the problem
is a MetaStock created one.

I will see how I can include a function in Analyst's DataServer to unlock
its data directories if locked and MetaStock is not running.


"Is there any way Analyst DataServer is able to discover the deleted
stocks and redownload them?"

Deleted symbols cannot be recovered. You can download the database
again but you will lose your SmartChart files.

Due to Equis' design, SmartCharts are saved according to the data
filename - the "nnn" in Fnnn.dat or Fnnn.mwd. Since the Fnnn.dat
and Fnnn.mwd can be different (for the same symbol) from user to user
and user with our database, therefore any reference to the "nnn" of the
filename cannot be synchronized. If only Equis had done the Smart thing
of using symbols instead of the "nnn" for the SmartCharts. Symbols will
always be the same.



This post has been edited by csk on Jan 31 2007, 10:20 AM

Hi CSK,

"Hist Adjust" can only adjust stocks one by one? Sometimes when i come back from holiday, i have a whole list of stocks to adjust for split/bonus etc. and usually i forget which is the last one i adjusted....................... :blink:

Dear csk,

I accidentally deleted some stocks while using Metastock.

Also, there was once when I clicked on "Deleted Expired Symbols" and some stocks which have not expired were shown. The stocks were deleted anyway and I could not recover them later.

Is there any way Analyst DataServer is able to discover the deleted stocks and redownload them?

The SGX website has not been stable lately. Their listed company information part
of the website was not available intermittently. This will be obvious when you use
an Internet browser.

When this problem occurs, the website will return a HTTP error code 503. This is
according to the HTTP protocol and standard.

However, some 3rd-party software firewall eat up (block) this HTTP error code
from being returned to the calling program, in this case, Analyst'sDataServer.
3rd-party software firewall were also been known to cause other problems for me
in the past although now I cannot remember exactly what they were or how it
happened. Often times, removing the 3rd-party software firewall solved the
problems.

Now, I use the hardware firewall on my Linksys router and the Microsoft Win XP
SP2 built-in software firewall without problem. HTTP error are passed through to
the calling program like they always should.

Lately, I received a few feedback of error "35756 Unable to complete request
Download File". This error message is thrown in the module, Download File, when
the program fails to get a response after a while of waiting BUT only if the HTTP
error code is eaten up by 3rd-party software firewall.

The program first get the HTTP header before attempting to download. Somehow
some firewall do not let that header through. Now I will trap that 35756 error and
display a message to show the correct problem instead of users thinking that is an
error somewhere.

If those 3rd-party software firewall has acted correctly, shown below is the error
message displayed by Analys'ts DataServer when the SGX website has a
intermittent problem or is down for maintenance in the weekend.

I caught the problem again this morning while testing version 3 of Analyst's
DataServer.

(Posted Image)

I provided more worldwide stock indices than the paid subscription data vendors.
Some may wonder why? They don't trade those markets anyway. In a way this
is true. But I find it very useful to know what the rest of the world is doing, at least
the major ones.

One stock index that stands out for me in the past and right now is the Swiss
Market Index. I have seen it leading the world. I show here its chart.

Notice the price action from mid-Mar2006 to early-May2006? Study it and what
after that. I had wanted to write about this index in Apr2006 but did not.

Now, look at the price action again this month, Nov2006. I am watching and very
eager to see how it forms the remaining part of it. I am eager yet be guarded
against having a pre-mature bias/prejudice that could result in a shut mind. A
mind that is shut to reality, in case I am wrong. In Buddhism, a shut mind is a
mental problem.


(Posted Image)

While doing these 3 charts, I realised that this data analysis should start after
Oct1987. The reason is that this annual human behaviour/psychology began
in Oct1988, the first aniversary of Black Monday.

In this case, the number for the earlier data analysis on the All-Sing Equities
should now be:

Total years: 18
Number of years advance: 15
Number of years decline: 3

Percentage of advancing years: 83.33% ((15/18) * 100)


Here are the data analysis for the DJIA, NASDAQ Comp and S&P 500:


(Posted Image)

(Posted Image)

(Posted Image)

CSK:

Can you do the similar "year end rally" test on DJ, Nasdaq & S&P ?

Thanks.

Ever since Black Monday, 19 Oct 1987, the market would always without fail talk
about or warn about a price decline in the month of October. This has now become
the "smart thing" to do. Or is it really that "smart"? I will prefer to go one step
further and be "smarter" then the "smart".

What effect does this annual "smart" human behaviour has? How can we make
use this annual "smart" human behaviour? In other words, how can we
be "smarter" then the "smart"?

The technical analysts of the past, they do a lot of data analysis. Not all their time
are spent entirely looking at technical indicators and coming with buy or sell. This
is the single most serious mistake a lot of present day technical analysts do. But
the pioneers spent the majority of their time with their data.

What I will show here is but a very simple type of data analysis.

Switch your focus back to the annual October ritual. It can quite safe to deduce
that because of the "smart" crowd, which by the way will include the fund
managers especially the young ones, their bearish bullets would be spent in the
period running up to 19 Oct. What this then means is that structurally, the market
is rid of a lot of selling pressure post 19 Oct. What this then means is that buying
power is likely to be greater than selling pressure post 19 Oct. Then what?

Then we look at the period from post 19 Oct to end-Dec. Why end-Dec? That is a
time that many squared or lighten their books for year-end accounting. To make
things simple I use monthly periodicity data using Oct and Dec month end data.
For better refinement, the analysis should be done on daily periodicity data on the
close of 19 Oct or the first market day after if 19 Oct is not a market day and
ending before Christmas when many people go on leave.

So here is the simplified data analysis code using EasyLanguage:

Here is the result:
Number of years advance: 15
Number of years decline: 4
Total years: 19

Percentage of advancing years: 78.95% ((15/19) * 100)


This simple data analysis on the All-Sing Equities Index shows that the odds
favour a general price advance from now till Christmas. I prefer to use the
All-Sing Equities instead of the STI because the STI components were "picked
and chosen" by their compiler while the All-Sing Equities covers a broader base.

At his point, I feel a bit uncomfotable about the market after Dec 2006. It kind of
look quite stretched. But nothing is better than to analyse the market without
pre-conceived prejudice when Dec 2006 comes.

Here is a graphic presentation of the data analysis:

(Posted Image)


This post has been edited by csk on Nov 1 2006, 01:35 PM

(Posted Image)

Once again, what use is this index? Affected so much today by just one stock,
Raffles Hldgs. But I don't see the problem is with Raffles Hldgs' special dividend
and capital distribution. Instead I see the problem lies in the way that the Index
computation is not adjusted forward taking into consideration what is happening
to Raffles Holdings. After all, this is what the Dow Jones and S&P do to their DJIA
and S&P family of indices. If I remeber correctly, the SGX Indices' computation is
made after consultation with a Professor from one of our local university. There is
very obviously a flaw in the design.

ps: The adjusted historical data for Raffles Hldgs and Natsteel (special dividend)
are already up on the website. Analyst's DataServer users can update their data
now. I may the first, if not the only one, to adjust for the corporate action in these
2 stocks today.


(Posted Image)


This post has been edited by csk on Oct 5 2006, 10:11 PM

I was asked about the adjustment I made to Metro historical data for their $0.03
special dividend ex-date 28Jul2006 and its effect on price chart analysis compared
to the the charts from the user's securities firm, specifically the trendline.

So I thought maybe I share it here on why I am doing what I am doing with
Special Dividend. In fact, few years ago in SI forum, Lion noticed the problem of
not adjusting with one of the SingTech listed company, I think ST Engg. At that
time I did not adjust although I see his point about the problem caused to the
charts. I have since decided to move away from the norm (and therefore made
progress) since the norm mess up the charts.

The reason I made adjustment is mainly because shareholders received the
Special Dividend in cash and this lower his original cost by same amount.

I know I am up against securities firms and paid subscription data vendors but
you can decide which is correct. Theirs or mine?

(Posted Image)

(Posted Image)


This post has been edited by csk on Sep 18 2006, 12:22 PM

thanks Cho and HC!

Hi Doozy,

I just sent you an email.

Cho


Note: You may now want to edit out your email address in your previus post.

Hi CSK,

I try to contact you via your contact page but it can't seem to work. Could you send me a mail? Thanks.

It is regarding subscription to the dataserver feature. Thanks!

Doozy

(HC note: doozy'z email address deleted)

This post has been edited by Hc on Sep 15 2006, 04:54 PM

Hi teachme,

The data will be from Yahoo! Finance and I am still in the process of looking
around Yahoo! Finance to see how best to provide their data in a form that is
usable.

Regarding your question on HKSE stocks (this should also apply to other
exchanges), I am looking to adjust the data using the program instead of just
loading Yahoo! adjusted data. There are two types of data from them; one is
completely unadjusted and the other is adjusted for all dividends and stock split.
What I am trying to do is to adjust the data just for stock split.

For unclean data, this really depends on what is unclean. Sometimes there are
spikes in the data and if the source is the exchange then Yahoo! cannot do
anything about it. Some vendors may intentionally remove the spike but then this
will not reflect the real thing. While to the eyes, the data without the spike may
looks like clean while the one with the spike looks unclean, reality is the spike is
clean while the no spike is artificially adjusted, therfore not clean.

For open with "0" as the price. This very much depend on the data vendor that
Yahoo! get their data from. In some quarters, there is a practice of putting some
prices into the day right from the start of trading session. If nothing is traded, then
the open remains at "0", the high is the highest bid, the low is the lowest offer, the
close stays at "0" until there is a trade or made to be the previous day's close if
nothing traded at the close of the day.

For data vendors who practice this, they are usually closed end vendors in that
they want to tie you to their charting software. Their proprietary charting software
(usually inferior) will skip days with "0" open and will not plot the data. This is a
sales trick to hook customers. The good TA software you get in the market do not
do this. Example, MetaStock and TradeStation 2000i.

If Yahoo! happens to have contracted with such vendors for some of the
exchanges then I have to filter the data accordingly.

For short history, this depends entirely on Yahoo! historical database. Their
emphasis is on US market and they carry very long history for them. For other
exhcanges, the data may start from 2000.

Actually, I want to do more than just price data. There are a lot of info on Yahoo!
but unfortunately mostly for US stocks. I want to do things that presently are not
available.

Back to HKSE data. I just ran the code snippet on stock split info for 1460 HKSE
symbols. There were only 5 stocks with stock split history. Not sure yet whether
this is correct.

Yesterday, after receiving the email below, I ran the same code snippet on 331
JKSE stocks and found only 1 with stock split info. Again I am not sure whether
this is correct.

I will need to load all the data and go through manually one-by-one to check. Not
really efficient but I ccan't think of other ways.

Dear csk,

Thanks for adding value to your existing DataServer customers.

Besides US stock data, may I know if your new product will include stock quotes for HKSE, adjusted for special events like stock split?

I guess your data will be from Yahoo. Will there be limitations like unclean data, data with short history (maybe at most 1 year)?

Anyway, I will be looking forward to your new product.

There have been requests for stock data other than SGX's. This is also something
I wanted to do so that the product can leave Singapore's shore. Well, in fact it has
already went to Malaysia, Australia and New Zealand. But so far only SGX data.

Products that rely on Yahoo! data are already available, even some subscription
data vendors in Australia are doing this. But I want to do it differently from what
is currently available. I want to do the stock split, hence price, hence volume,
right. I will not divulge too much. I remember last year I mention about quaterly
and yearly chart in Analyst's TurtleFarm. Well coincidentally after that, a local
data vendor implemented quarterly and yearly chart capabilities into their
product.

I am testing the codes for downloading US stock data from Yahoo! Eventually,
this will include all other data available from Yahoo!. The code logic is basically
the same. Since this is targeted also for foreign users who may not have an
interest in SGX data, this is very likely to be a separate peoduct. Existng users
of Analyst's DataServer can get this at a very generous discount. They will be
informed prior to the release. But only for existing users at that time before
release. The generous offer will not apply to new users after the release.

This is a screen shot of the code snippet being tested. It is not what the product
will look like.

(Posted Image)


As promised, Warrant Calculator and Crocker Price-Volume Chart will be free
to existing users at the time of release. Both will be released at the same time.
They have been taken out of Analyst's DataServer and reworked as full
independent products. Upon release, they will be stand-alone products.



This post has been edited by csk on Aug 31 2006, 10:26 AM

"(1) is the start date critical ? will it skew the decision making process ?"

As mention earlier in my previous post, the start date of computation of CADI is
not important. Since it will always start with 0 therefore the CADI value number
on the Y-scale is not important. Analysis is done on the shape of the CADI and its
primary use is to look for divergence, just like with a momentum indicator.

The two chart examples below show the CADI being computed with different start
dates and although the scale values are different, the shape of CADI in both chart
are exactly the same.


"(2) since SGX CADI is not very useful, where can we get the correct CADI for
other markets ?"

If the pecularity can be discovered (a lot of work needed in the research and the
maths) then it can be useful. I am not familiar with sources for the CADI for other
markets except USA. You will find in this link:

http://unicorn.us.com/advdec/

Do note that the author mention about differences from different sources.


(Posted Image)

Repost:

(Posted Image)

(Original thread: http://z7.invisionfree.com/ChartistsUnited...post&p=2610802)

Hi CSK,

Thks.

Looks to me that CADI is superior because it requires lesser input (no unchanged data) for the same result!

My question,
(1) is the start date critical ? will it skew the decision making process ?
(2) since SGX CADI is not very useful, where can we get the correct CADI for other markets ?

I am interested in indicators other than "price indictors" (have tons of them and I believe I will never ever have any use for all of them).

For example volume, time, volitility etc ? Forummers please share. Thanks

The Bolton-Tremblay and Cumulative Breadth Indicators produce quite similar
shape as the CADI (AdvDec Line). The only difference is their computed value but
this is not important. In fact, the computed value of CADI is also not important as
it really depend on when you start the computation on the chart. It will always
start with zero on the first day of computation. Similarly, Bolton-Tremblay and
Cumulative Breadth always start with zero.

Their uses are bascially to look for divergence between the stock market index and
the indicator, very similar to the divergence found in many othr technical indicators
like MACD, RSI, Momentum, Rate Of Change, etc.

However, because of the pecularity of the CADI on SGX indices where it show a
long-term tendency to always slope downwards, the intended use of CADI then
because somewhat of a challenge since it tend not to who up bullish devergence at
bottoms and always show bearish divergence. Note: this was discussed in the
CADI thread (at this link).

As I am not familiar with MetaStock formula language, I list here the respective
formula in TradeStation EasyLaguage for you to create them in MetaStock.


Formula for CADI:

Formula for Bolton-Tremblay:

Formula for Cumulative Breadth:

Note that Cumulative Breadth divide the (Adv-Dec) value by the sum of
(Adv+Dec+Unch). As compared against CADI, this should have normalized the
computed values but did not. This should have taken care of the situaution where
there were fewer stocks listed in the past and more and more stocks being listed
as time goes by. However, normalization failed here as far as the pecularity on
SGX is concerned.

I did not show the Hughes-Russell-Schultz indicator because when I was
programming into EasyLanguage, I realized that the formula was exactly the same
as Cumulative Breadth.


(Posted Image)


This post has been edited by csk on Aug 13 2006, 08:29 AM

Hi HC,

Since they were in the initial todo list and had to be dropped, they were always on
my mind. If I can find an efficient way I will put them into the package.

I have actually thought of a method where there is no need to loop, load and
check but its implementation is problematic. It requires users to update the data
in strict accordance to the date sequentially without skipping or re-updating. These
requirements are common not to be met in the course of normal updating so
implementing it means there will be errors in the numbers.

Although Analyst's DataServer keeps track of dates and prompt correctly, it is
common that sometimes users may re-update and/or skip dates already updated.
The program was designed to allow for this flexibility to ensure data integrity. For
this reason, the faster method could not be implemented.

Don't worry. I know you are sharing your thoughts and ideas. Glad to hear you
are also happy with the product.

Please be assured that once I find a way to do it efficiently, I will "Just Do It!".


This post has been edited by csk on Aug 12 2006, 08:36 AM

CSK:

Yes, I am aware of the time and effort commitment if data will to be downloaded from your server. Given the fact that Analyst's DataServer is just one time fee, I can see that asking for the perpetual availability of this processed data is unreasonable.

Looks like we have to settle with:

1. Just leave it there.
2. You charge a subscription fee (say half yearly) for provision of this additional processed data (and more to come, if there is any)
3. Do a separate program (charge separately if necessary) for existing Analyst's Dataserver user that is able to "generate the whole New High/Low data file" and "Check last update and append new New High/Low data", extracting from the already downloaded TAR-Database and write the New High/Low file. This can be a separate .exe file that user have to run after daily download.

I doubted the market of option 2 & 3, thus I think we have to live with option 1.

(PS: I just sharing my thought, not complaining :). I am a happy user of Analyst's DataServer.)

Hi HC,

Due to the reason of the effect on daily update processing time, the 52-wk High
and 52-wk Low were taken out of the todo list during the codes implementation.
Right now, it it not in the feature list because the situation has not change.

Since the Analyst's DataServer is a low cost alternative (and providing
tremendous savings in the long run), the idea is to have the work done by the
software automatically. Where it cannot and is absolutely required then I
implement externally, for example the Corporate Action.

To make these two data available as a separate download might cause some
complication in the future. I want to avoid a situation where there may be friction
due to not making the data available everyday promptly and not willing to accept
the longer update process, if you know what I mean.

If there is a more efficient way of putting these data into the package I will do it.
For the moment, I should not do anything to put unnecessary overhead onto the
daily process and upset users.



This post has been edited by csk on Aug 11 2006, 10:59 PM

CSK:

Ref: SGX New High and New Low (52-week High and 52-week Low)

Just wish to voice my personal opinion: I don't favour such extended time required just to churn out data of new High/Low. As this is going to be done on a daily basis, and I doubt many people will make use of this data and look at it daily. I think the overhead does not justified having this data generated daily.

However, I do see some use of this data, and I am wondering if this can be done as a seperate download from your server where the computation is already done on your side, user just download and update metastock data files will do.

This optional download, can be just "Download the entire latest New High/Low Data File" if the implementation of a option "Download todays' new High/Low" to keep track daily update at the user side is troublesome. With this arrangement, users who need this piece of data will do it seperately (or optionally), while those who don't don't have to incur additional data updating time yet still preserving their opportunity to have them later when required.

Thank you for your kind consideration.

TRIN (Arms Index)

TRIN stands for short-term TRading INdex and is also commonly known as The
Arms Index. It was invented by Richard W. Arms, Jr. in 1967 and has been very
popular in the USA ever since. It has its own ticker - TRIN. Richard W. Arms, Jr.
also invented the EquiVolume charting.

Formula:
TRIN = (Adv/Dec) / (AdvVol/DecVol)

TRIN calculates the ratio of the Adv/Dec Ratio to the AdvVol/DecVol Ratio. It
shows whether volume is moving into advancing or declining stocks. Values below
1.0 indicate more volume in advancing stocks, while values above 1.0 indicate
more volume in declining stocks.

Popular use of TRIN is that Values over 1.2 suggests oversold conditions while
values under 0.7 suggests overbought. But these values need to be adapted to
different stock market. To smoothen out the index, 10, 20 and 55 days moving
averages are used. Its use can be more complex than this.

The Arms Index is a book itself so the common explanation may be just a scracth
of the surface. Unfortunately, I don't have this book so I can't say much.

Link to the book:
http://www.amazon.com/gp/product/188327215...8096615?ie=UTF8

This is very sad. Inspite of its popularity in the USA, no data vendor or newspaper
in Singapore bother to make it available here. Perhaps it may be because data
required for its calculation are not available, namely Adv Volume and Dec Volume.
But then are they really not available? Or is it no one bother?

So instead of waiting for someone to do it, I do it. Just do it! like Nike says. It is
bad to see Singapore stuck in the backwater because no one can or bother to do it.


(Posted Image)



This post has been edited by csk on Aug 12 2006, 09:03 AM

SGX New High and New Low (52-week High and 52-week Low)

52-wk High and Low were in the initial todo list but this is a bit of a problem. It is
easy tocompute but to compute it everyday will mean the updating proccess will
take longer. I am not sure whether users will accept the longer updating time.

Since this information is not available, the program will then need to loop through
all the data files, loading the last 1 year of data and check.

Right now, it takes about 2.5 to 3.5 mins (depending on processor speed) to
update one day data including the Market Breadth computation. Some have
security software running (especially anti-virus) that may interfere with the
updating since files need to be open and written to, stretching the updating time.

Looping, loading and checking the highs and lows will make the updating time
much longer.


HKSE data

The data from HKSE is missing the Open. I believe it is still the same. The task
mentioned earlier about tracking and follow up on Corporate Action is still a
problem.

This post has been edited by csk on Aug 11 2006, 12:18 PM

Hi teachme,

I will be doing the TRIN and the other four indicators mentioned. Adv and Dec is
also used for Cumulative Advance Decline Index (CADI), or popularly known as
the Advance/Decline Line in the USA. See the link to the CADI thread I gave in a
previous posting.

I had originally wanted to compute CADI like all the Market Breadth data so that it
goes into a file of its own. But it is a cumulative number that depends on previous
day/s data and once it is wrong on one day it will then be even more wrong after
that. Since there is no way to ensure that users update the data in proper date
sequence therefore there is no way to ensure that the CADI can be computed
accurately. For this reason, I take CADI out of the database.

The formula for CADI is:

CADI(today) = CADI(prevday) + (Adv - Dec)



This post has been edited by csk on Aug 11 2006, 11:58 AM

Dear CSK,

Thank you for your elaborate replies. It is even better than the help file provided in the DataServer.

There is still one more item TRIN under Market Breadth which is not covered. Would you mind to talk about that too?

It will be great if you could add the number of new high and new low in SGX to your DataServer.

Does HKSE provides free EOD data like SGX? If yes, maybe you can try adding it to your DataServer.

UpDown Vol Spread

Formula:
UpDown Vol Spread = Adv Volume - Dec Volume

The UpDown Vol Spread (AdvDec Volume Spread) is the difference of the sum of
the volume of advancing stocks and the sum of the volume of declining stocks.
The basic logic is similar to its cousin the AdvDec Spread with volume as added
input. Like its cousin, this data is also very erratic.

Caution: This data is likely also affected by the many stocks in SGX that trades at
few cents. It is common that such low price stocks can have high volume but
relative low dollar value of transaction. Because of this, spikes in this data can be
more exaggarated or the data can be more erratic.

(Posted Image)



This post has been edited by csk on Aug 11 2006, 08:31 AM

UpDown Vol Ratio

Formula:
UpDown Vol Ratio = Adv Volume/Dec Volume

The UpDown Vol Ratio (AdvDec Volume Ratio) is the ratio of the sum of the
volume of advancing stocks to the sum of the volume of declining stocks. The
basic logic is similar to the AdvDec Ratio, but with volume as added input.

Caution: In SGX where there are many stocks trading at prices of few cents, it is
easy these stocks can have high volume but not high dollar value. A stock at
$0.02 with a volume of 20,000,000 requires $400,000 but a stock at $2.00 will
require a massive $40,000,000 for the same volume of 20,000,000. Because of
this, spikes in the data can be exaggarated.

(Posted Image)



This post has been edited by csk on Aug 9 2006, 09:25 PM

AdvDec Spread

Formula:
AdvDec Spread = Adv-Dec

The AdvDec Spread is simply the difference between the number of advancing
stocks and the number of declining stocks. The values oscillate around zero. Its
use is quite similar to AdvDec Ratio - extreme highs indicate overbought
conditions while extreme lows indicate oversold conditions. The Ratio is not as
erratic as the Spread.

(Posted Image)



This post has been edited by csk on Aug 9 2006, 08:41 PM

AdvDec Ratio

Formula:
AdvDec Ratio = Adv/Dec

The Advance/Decline Ratio is the ratio of the number of advancing stocks to the
number of declining stocks. The result is easily read. A value of two means that
there were twice as many advancing stocks as declining stocks. Values below one
mean that there were more declining stocks than advancing stocks. For example,
a value of 0.5 means that there were half as many advancing stocks as declining
stocks.

While many go estatic when they see a big number stocks advancing, it is
unfortunately that it also means that a lot of people have already put their money
into those stocks. With fire power used up, according to nature's law, there has to
be setback. From this overdone condition, an oppostie move can be expected
even if it is a short one of only a few days.

For this reason, the AdvDec Ratio is usually used as an overbought/oversold
indicator, but with caution I must say. Cannot any how shoot. From the chart
below, you can see that when there is a spike in the AdvDec Ratio to above 2.5,
the market rise tends to pause or correct. So in a rising market, this Ratio can
offer an advance warning and a precursor timing tool. Since it is quite difficult, if
not impossible, to so short strategically, I shall not talk about the dips in the Ratio.


(Posted Image)



This post has been edited by csk on Aug 9 2006, 08:09 PM

I have actually explained the reason in yeaterday's posting:

"In the days immediately following 9/11, quite a number of stocks were not traded.
The SGX used the last traded price of the previous day/s as the close for the days
that the stocks did not trade. This is their standard practice and is necessary for
the purpose of marking-to-market. It did not mean that the stocks were
unchanged. If they had traded where would they have closed? Then again how
about those stocks that are suspended indefinitely? Unchanged or the value is not
there at the last price any more? This is an important consideration especially
when Unch is used in calculating market breadth indicators like:

Schultz A/D
Cumulative Breadth
Bolton-Tremblay
Hughes-Russell-Schultz Breadth"

Because the method to include non-stocks in the numbers is wrong in the first
place, I do not compute the incorrect numbers. Like you, but from the other side,
I could not do the testing for margin or error. Although I can easily write the
codes to compute them, there is really no benefit. Since the basic rule is not
followed therefore the approach is flawed.

I give you an example - the inventor of the RSI taught that it is a momentum
indicator. But over years, many people throw in their own interpretation that the
RSI is an overbought/oversold inficator. They are plain wrong in their
interpretation. The worst thing is, these people propogate their wrong
interpretation all over the Internet.

Does this make the RSI as such? In the same way, the interpretation of the
Unchanged data.

The Unchanged is pretty much neglected in present days. Maybe people are
attracted to rising stocks and despair over falling stocks. It is true no use can be
found in the present days.

I have indicated four indicators, in yesterday's posting, that uses Unchanged data.
They are found in the CompuTrac/Apple][ manual. I will create the indicators in
TradeStation and show them but will need time so the examples may not be so
soon.

By the way, a number of indicators that were in CompuTrac/Apple][ were left out
of CompuTrac/PC when development shifted. They were also not found in other
TA software. Maybe the inventors of the nidicators preferred it this way, I am
guessing but I can recreate those indcators left out.

Below are the Adv, Dec and Unch data for last night US sock markets taken fom
Yahoo! Finance. Notice the Unch is 139 and that Total is only 3402. I am very sure
there are more than 3402 stocks listed on the NYSE. I know because I created the
symbol lists for Analyst's TurtleFarm.


(Posted Image)


Yahoo! rightfully discard not just the non-traded stocks but also the non-stocks
symbols. US Stocks have long chains of call and put options on them listed on the
exchanges. To include them would means millions of symbols, isn't it.

Structured warrants is just another form of stock options but structured in a way
that makes it advantageous to the original writers. Unlike US stock options where
you can write (short) to take advantage of high Implied Volatility (IV), in SGX
structured warrants, you cannot write (short). Only the original issuers which
happen to also be the market makers can. And they put them high IV monies
(stupidly paid by people) into their own pockets in the first instance.



This post has been edited by csk on Aug 9 2006, 12:39 PM

Sorry for the repost,

I did not specify clearly my question (Thanks to late night drinking.. Cheers Singapore)

I do not have the correct ADV-DEC data. Therefore I could not do the testing.

But I am interested, how much margin of error will the wrong ADV-DEC data introduce.

It seemed that the Unchanged data is up to interpretation, unfortunately many data vendors are using total unchanged data, how important is it? And I have never come across indicators that use the unchanged data, could you highlight the usage ?

Appreciate for your time,
Thanks

Hi CSK,

I'm new to the ADV-DEC stuff.

You have posted

However 100+186+191= 477.

Did I miss anything ? Thanks

addition:
And i reread and reread the post, i guess for unchange stocks, you have included only traded stocks, why ? thanks


This post has been edited by Undead on Aug 9 2006, 08:16 AM

I made a posting in the CADI thread last year 3-Sep-2005 on the problem I cited
earlier about the wrong Adv, Dec, Unch data provided by all data vendors.

The link to that posting is (hopefully the link work, if not look for 3-Sep posting):
http://z7.invisionfree.com/ChartistsUnited...dpost&p=2684509

In that posting, I mentioned that the numbers I get are quite close to that on
Yahoo! website. I just did a check again and they remain close. See screen
capture below. However, Yahoo! is not a data vemdor and they do not provide
such data in historical form. At least not I know of.

(Posted Image)

I will begin the various Market Breadth data in Analyst's DataServer and their
uses with the explanation of the RAW data form and how they are derived.

The raw data form are:

Adv
Dec
Unch
Adv Volume
Dec Volume


Adv

This is the number of stocks that advanced for the day. The formula is the sum of
all the stocks where Today Close > Previous Close.


Dec

This is the number of stocks that declined for the day. The formula is the sum of
all the stocks where Today Close < Previous Close.


Unch

This is the number of TRADED stocks that closed unchanged for the day. The
formula is the sum of all the TRADED stocks where Today Close = Previous Close.


Errors found in ALL data vendors

Here, I have to say that presently ALL data vendors that offer SGX data package
are providing wrong Adv, Dec and Unch data to their customers. You can easily
verify this. It is very easy. Check the data below against data you find from
elsewhere.

As of yesterday (7-Aug-2006) close, Analyst's DataServer counted 724 stocks.
The rest (528 non-stocks) are either Trust (including REITs), Rights, Bonds,
Warrants and of course Structured Warrants. Now, 528 is a very big number given
that there are only 724 stocks. Since the definition for Adv, Dec and Unch specify
only stocks therefore Analyst's DataServer is programmed correctly.

The coumputed numbers for 7-Aug-2006 are:

Total: 724
Adv: 100
Dec: 186
Unch: 191

All the data vendors are sure to give higher numbers for Adv, Dec and Unch. For
Total, they may give 1252. The reason? They include ALL symbols. As an example
of the harmful effect of such wrong computation, let's just use CapitaLand as an
example. One mother stock, how many structured warrants? See this screen shot.


(Posted Image)


From the above computed numbers, the number of stocks that was NOT traded
was 247 (724-100-186-191). This number is correctly discarded out of the Unch by
Analyst's DataServer. This method of computation is important.

In the days immediately following 9/11, quite a number of stocks were not traded.
The SGX used the last traded price of the previous day/s as the close for the days
that the stocks did not trade. This is their standard practice and is necessary for
the purpose of marking-to-market. It did not mean that the stocks were
unchanged. If they had traded where would they have closed? Then again how
about those stocks that are suspended indefinitely? Unchanged or the value is not
there at the last price any more? This is an important consideration especially
when Unch is used in calculating market breadth indicators like:

Schultz A/D
Cumulative Breadth
Bolton-Tremblay
Hughes-Russell-Schultz Breadth


Now for the last two raw data form. They are straight forward once you
understand the above.

Adv Volume

This is the summation of the volume of all the stocks that advanced for the day.


Dec Volume

This is the summation of the volume of all the stocks that declined for the day.


This chart below shows the above raw data plotted against the SGX All-Sing Index
for visualization of how they look like. They are usually plotted as histogram in
their raw form.


(Posted Image)


I will follow up with the others and more...





This post has been edited by csk on Aug 8 2006, 03:38 PM

Hi teachme,

I am very glad you like Analyst's DataServer. Thank you for your encouraging
words about the product.

I will answer your questions on the Market Breadth data in separate post. I need
to first go get some charts ready and will revert. Description was not included in
the Help file as I thought they are quite standard but I will include them in the help
file in the coming version.

On the questions of HKSE data, I have been asked similar question. The biggest
problem is keeping track of Corporate Action and adjusting the historical data
accordingly. You would have notice that we do the adjustment for SGX data very
timely (unless the stock is inactive immediately on and after ex-date) and
accurately. This can be a massive and tedious task if other exchanges data are to
be handled the same way that the SGX data are being handled.

I have thought of using Yahoo! data as this may be the easier solution but Yahoo!
do not provide for all corporate action, eg. bonus issue and cap reduction. They
also do not list stock splits as proper files so quite tedious manual searching is
required. The split info is needed to do proper historical adjusment for the volume.

One method is to do what TradeStation does - store the data unadjusted and keep
a split dates calendar. When the data is loaded, check against the split calendar
and adjust the data on the fly durng the loading process. However, my
understanding is no other software do it this way so ruling out this feasibility.

The alternative is to have ad-hoc download of the entire historical data when
needed but I am quite sure some may not like it this way. Being able to include
other exchange data is actually a good thing as this would bring this product
outside Singapore to a bigger market. I have not found a reliable solution yet.



This post has been edited by csk on Aug 7 2006, 11:54 PM

Dear csk,

I am a satisfied user of your product, Analyst's Dataserver. Thank you very much for this great product!!

There are a few questions on this product that I could not find the answers in the help file.

The following value-added information were DataServer but no explanation was provided for them. For example, how they are calculated and better still, how to interpret them.

- Market Breadth data
- SGX TRIN.
- SGX AdvDec Ratio.
- SGX AdvDec Spread
- SGX Adv Volume.
- SGX Dec Volume.
- SGX UpDown Volume Ratio.
- SGX UpDown Volume Spread.
- SGX Adv. New!
- SGX Dec. New!
- SGX Unch.

Would you care to explain them here?

Take your time to reply as I am aware that you are down with flu recently.

By the way, do you have future plans to provide EOD data for HKSE? I have been trying to find a similar product like yours for HKSE, but there are none as good.

You did a great job with DataServer!!

Most TA software, at least all the popular ones, do not allow you to open more
than one instance of the program. I know TradeStation won't allow you to do this.
So too MetaStock. You can try to launch more instance of the program from the
Start menu but all you get is the currently opened program being presented to
you.

I find this not acceptable. MS Excel allows you to open as many instance as you
like and as your computer RAM and Windows resources allow. So too MS Word,
MSIE. The only exceptin to this is when the software did to control a central
database or commn set of files, like Outlook, then this is understandable.

So except for Analyst's DataServer which manages its own central MetaStock
format database, all the other analysis software will open more than one instance.
This allows users with more than one screen to be able to have the software
opened on all the screens if he/she really likes the software so much.

Once, a TradeStation user asked me, "I want this chart on this chart full screen on
this monitor and this other chart full screen on that monitor." I had to answer
him, "It is not possible."

With the software I write, this is definitely possible. Here is a screen shot of two
instances of Analyst's TurtleFarm opened. If you have two screen, you can drag
one over and maximised it. I can't do such a screen shot so the next best is to
show them in one screen. The maximum intances is only limited by RAM and
Windows resources, not by program design.

(Posted Image)



This post has been edited by csk on Jul 14 2006, 03:34 PM

There were two problems I encountered with the 3rd-party library when reading,
therefore trying to load, dates earlier than 1910. Initially, I tried various
workaround but after some time I had to accept that I could not provide all the
workaround for all the possible scenarios I could think of and some which I might
not have thought of. So, eventually I went back to the vendor and they fixed the
problem at source. Phew.

So from the next version of Analyst's DataServer when it ships with the new
updated library, the DJIA historical data will be provided back to 26-May-1896, the
first recorded data of the DJIA. I believe no where else, no other vendor provide
this far back. I may be wrong but I don't think so.

Analyst's TurtleFarm will come with the new version of the library and so will read
the data without problem.

RafflesPlace has been revived - it will read CompuTrac/MetaStock data but it is
also targeted for users who prefer to load Yahoo! data on demand and thus prefer
not to manage data locally. Of course, because of the unreliable SGX data on
Yahoo!, CompuTrac/MetaStock format SGX data is still a must.

Also in the pipline is Analyst's RadarScan. Although MetaStock Explorer can do
scanning/filtering, but there are enquiries on such a software.

All software will now sport the new look toolbar. See screen shot below of Analyst's
TurtleFarm.

(Posted Image)





This post has been edited by csk on Jul 14 2006, 02:29 PM

The most difficult part of the programs' conversion to XPTheme-aware is
manipulating the text color of individual ListView contents. For example, I want
certain fields to be blue if the price is up, red if down and black if unchanged.
Users of Analyst's DataServer will notice this in the EOD Snapshot.

Doing this without XPTheme-awareness is easy and straight forward. But to do
this when using the ListView from the appropriate Windows Common Comtrol for
XPTheme-awareness left many of the features to be programmed via the Win32
API. And getting it to display with colors in dividual cells is one hell of a steep
learning process and a lot of digging into Win32 API messaging and subclassing
and countless program hang and crashes.

Despite the difficulty I have to make it work otherwise programs will look very
aged especially when Windows Vista is released. It is like you go to an older
bulding where the walls and floors are covered with mosaic and a newer one
where the walls and floors are coverd with marble.

Some screen shots below. Notice the blue and red colors of the price make it easy
to view visually. This is where the Win32 API gets difficult but it is finally done.
Most important the programs now use "marble" instead of "mosaic".

Notice the native Win XP theme presentation of the tabs and column headings and
how they change with the change in the XP theme. Notice also the different
appearance of the status bar at he bottom of the program window. There is no
effect with users still with Win2000, ME or 9x which do not have XP theme. They
still get the "Classic" look, ala mosaic.

(Posted Image)


This post has been edited by csk on Jun 1 2006, 04:50 PM

To test the loading time required for Yahoo! data and stability, I wrote a simple
program to do this. I tested with loading the 30 component stocks of the DJIA.
Yahoo! has the longest historical data for US stock. For other country, they have
much less.

Here are the number of DJIA stock and the year data are available from Yahoo!

10 - 1962
9 - 1970
1 - 1972
2 - 1977
1 - 1982
1 - 1983
4 - 1984
2 - 1986

Tests were done at different time fo the day and different time of the week using
SingNet 512K broadband. Usually, the time taken to load all the 30 stock data is
approx 4-1/2 minutes. Today, I get just under 4 minutes, probably because it is
a Sunday.

Loading speed will of course be very much faster in the US and Western Europe,
my perception is could be from 1 to 2 minutes. I am, of course won't be surprised
it it us just a snap of the fingers. Since this program is targeted for world market,
I think the result I get is acceptable.

(Posted Image)

Like it or not, Yahoo! data is popular because it is free. Some of the data you can
use, especially the more popular ones like US, Canada and UK and some others.
But getting the complete and up-to-date symbol lists is a real challenge and very
time consuming.

Through my working with the OEM version of Telerate TradeStation, I find that the
feature to load data on demand direct from a database server is very useful. It
obviate the need to store data locally on the computer and therefore also obviate
the need to ensure data adjustment are made promptly, especially when you don't
have the info or just simple not possible to track all the dividends, splits, etc of all
the stocks in the universe. Everything is left to the database people. When you
load the data on demand, you load the latest set of data available.

You of course do not have this luxury with SGX data since Yahoo! SGX data is
not fit for use at all. Two decimal places and many non-adjustment of data for
some corparate action the main grouse.

Analyst's TurtleFarm now has the added feature to load Yahoo! data (in addition to
CompuTrac/MetaStock) for the charts and save the charts on the computer. Later
when these charts are opened, it will load the latest Yahoo! data, input all the
saved parameters, calulate the Turtle System and display the charts with exactly
the same display properties when it was saved. Quite a lot of work was done to
achieve this functionality.

(Posted Image)

Analyst's TurtleFarm is also now XPTheme-aware as the above screen shot shows.
Quite a fair bit of codes were reworked to use Win32 API programming and
Subclassing. In some situations, the program codes have to intercept and act on
messages sent between Win OS and the program.

This next screen shot of the program was captured and saved from within the
program. The dialog boxes below it shows the two image capture possible -
another idea from TradeStation and SuperCharts which has this capability since
way back in 1993. Supported file formats are png, jpg, bmp and gif.

(Posted Image)



This post has been edited by csk on May 28 2006, 04:16 PM

United Overseas Land Ltd is now changed to UOL Group Limited. Analyst's
DataServer uses the information provided in the SGX source to automatically
make this change to the database when updating the data. This change is subtle
but in the past names had been changed to completely unrecognisable. Since the
source has to be the SGX assigned name, the program uses whatever is available
to handle all the necessary. New listings, stock IPO, rights, new structured
warrants etc., are created automatically with the SGX assigned name and symbol.

However, one frequent support feedback is that the use of SGX's assigned
30-character name is giving problem when changing stock within the MetaStock
chart program. The reason is that the full name is not shown in its entirety but
truncated. This actually is an Equis programming design flaw.

To give some background to the MetaStock format. The format is originally the
CompuTrac format which was released into the public domian by CompuTrac in
the early 1980s. When Equis released MetaStock sometime in the later half of the
1980s they took the CompuTrac 5 (DHLCV) and 7 (DOHLCVI) fields format and
unshamefully called it the MetaStock format.

The CompuTrac format has a limit of 16-characters for both the Name and
Symbol. Some time later, Equis modified the format by increasing the Name to
45 characters and adding the EMaster file to accomodate the change thereby not
touching the Master file. Since it is Equis that made the change, therefore their
program should logically be able to handle this.

SGX's 30-character Name is well within the MetaStock specification. This therefore
is not an issue and therefore cannot be the cause of the problem MetaStock users
are facing. The problem is poor user interface design by Equis! But this is difficult
for a number of MetaStock users to accept. To them it is an irritation and to them
the irritation is from the use of SGX's 30-character Name. But I want to show here
that this is not. My advice to them has been to feedback to Equis and get them to
change their user interface.

This is the why they are irritated by Equis' poor design (not by using 30-charater
Name). See this screen shot from MetaStock:

(Posted Image)


Now compare the poor design above to this one here which I punch out within
minutes. You will see immediately that the resolution to the irritation is really very
simple. Equis' programmers have to do is to W I D E N the dialog box. This is
example of a good GUI (graphic user interface).

(Posted Image)


It is not that Equis is not aware. It cannot be because it is they who make the
modification to 45-character. So they goofed. To further show that they correct
the problem, here is a screen shot of MetaStock Open dialog. See, no irritation.

(Posted Image)

The following five suggestions are from a user. They will be in the next version. All
will be available as options, i.e. users can set them to on or off:

1) Auto update of previous day SGX indces OHLC
2) Auto update of Yahoo! Asian indices
3) Auto update of Yahoo! latest FX rates
4) Auto deletion (backup optional) of expired stocks
5) Auto deletion (backup optional) of expired warrants

If all options are set to on then it become a one-click process.

Yahoo! ^BSESN will be grouped with Asian indices. Yahoo! US and Europe indices
update remains a separate update process due to different time-zone.

SGX stocks and warrants will be in two separate data directories.

Yahoo! made changes to some indices name recently. The change to S&P 500,
^GSPC, causes a parsing problem resulting in info going into the wrong fields.
This problem is solved in the next version.

In the present released version , the problem is only seen in the ^GSPC latest
price update and only happen when Yahoo! has not yet updated their historical
database for the US trading day (our early morning). Luckily, Yahoo! update their
US database quite early so an update around 8:00am or earlier will not have the
problem. But updates around 6:15am will have the problem. Luckily the program
code is fool-proof and a re-update later will correct the problem. This problem is
gone in the next version.



This post has been edited by csk on May 5 2006, 11:45 PM

I have been receiving questions on the adjusted history for corp action. Here are
extracts of two example questions and a reply from me:

Q1: What happen if I update the same stock for historical adjustment twice (say 2
for 1 split)? will the data be adjusted twice or will it be able to distinguish that
adjustment has already been made?

Q2: What happen if we do the adjustment ourselves manually (say for a split) and
then later click the update data using the one provided by the software?

A:
Let's take this hypothetical example:
1) Stock XYZ Ltd last traded at $6.000 last volume 10,000 just before ex-date for
1 to 2 stock split.
2) You manual adjust for the split. Data file now show last price $3.000 last
volume 20,000 (all history also adjusted).
3) You accidentally manual adjust again. Your data file now show last price 1.500
last volume 40,000 (all history also adjusted).

The adjusted file on our website will show last price $3.000 last volume 20,000 (all
history also adjusted). No matter what your data file is showing (1, 2 or 3 above
or any other), when you update the adjusted history from our website, your file
will then as a result contain data that is identical to those from the website.
Doesn't matter how many times and doesn't matter what you may have done to
the file previously. This ensure that the program feature will always work correctly
as expected and any error corrected automatically.

The process had been carefully thought through to make the process fool-proof to
cater to real world usage where possible scenarios were analysed and potential
data problem avoided. The resultant process is actually very easy to implement,
thankfully to my own data usage experience. It is so important that softwares be
written by users for users and not by programmers for users. There is a lot of
difference in the end products.

Very late in the day, Auric Pacific was traded:

16:56 1.180 1000 shares
17:12 1.260 1000 shares (pre-close)

The adjusted data has been uploaded and will therefore synchronise with SGX
lastest price froim here on.

Been testing to make my software theme-aware. I know I can easily do this with
Visual Studio 2005 but the problem with any version of the .NET Framework is that
compiled binary can easily be decompiled. This is very similar to what Java
programmers have been suffering since day one of Java.

Of course there are commercial Obfuscator that claim can prevent decompiling
and cost from few hundreds USD to thousands of USD. The reality is people are
coming up with new ways to decompile. The cause of this problem lies in the
strutural design of the .NET Framework.

So what I am doing is to go into low-level Win32 API programming and maybe
some subclassing and hooking to achive theme-awareness.

Here are screenshots of what I mean by theme (only apply to WinXP and later).
The program is just an empty shell with no code:

(Posted Image)

Of the 5 stocks due for corporate adjustment yesterday, 2 were not traded - Auric
Pacific and Hiap Moh. Even right now they are still not traded. As such, SGX will
continue to provide the last pre ex-date price every day. Adjustment can only
be made when there is trading activity to synchronise with SGX data. The other
3 were adjusted and uploaded yesterday afternoon.

There are 4 stocks due for corporate adjustment today and so far, Apollo Ent is
not traded. So similar to above, have to wait. The rest have already been done
and the adjusted history uploaded.




This post has been edited by csk on May 5 2006, 03:03 PM

This is a busy period for adjustment to corporate action. There were 13 adjustment
last week, 8 so far this week with another 10 the next 2 days for a total of 18 this
week. At last count there should be 10 next week with more to follow after that.

(Posted Image)

Thanks for your input.

When I approached the UOB bank branch manager some time ago for clarification, she said that the account can accept funds from any bank and through any of the recognised fund transfer method which include Internet transfer. She could not understand how some Internet transfer failed because as long as the payer bank give the correct account info, the transfer will not be rejected by UOB.

Direct credit card payment via PayPal is also available now on the website. This is readied now in preparation for the launch of Analyst's TurtleFarm within this month, with interest coming from the West based on website statistics.

so far, never come across bounced transaction across all banks. the only thing is the 2-3 working days required for interbanks.

anyway, will be going thru all my stuffs before sending the cheque and start the ball rolling.

Hi Silverbullet,

"CSK, can use internet bank xfer or not."

Thanks very much for your interest. Yes, of course but it is good if I share my experience with Internet banking.

There might be problem when transfer is from one bank to another bank. We have experienced such transfer failing. Behind the scene, Internet transfer is still done manually by the banks so it will take about two to three business day to be effected or transfer failure to be made known. In the meantime, the payer bank will debit the payer account immediately and the funds will float for this period until it is credited into the payee account (successful) or credited back into the payer account (transfer failed) after the two to three business days.

There may not be such problem if the Internet transfer is between accounts of the same bank.

I would like to know of your or anyone else experience with Intenet banking. I appreciate any info. Usually when I thought I be careful to explain in case there might be the same problem we experienced previously and to prevent any misunderstanding that we received payment but didn't deliver, I felt I might have scared them off.



This post has been edited by csk on Apr 6 2006, 04:00 AM

CSK, can use internet bank xfer or not. Hate to use check .. hahaha.

i suppose Technical Analysis & Research's Data analyst software is the best?

There have increasingly been more and more structured warrants listed. The
issuers are even going after stocks that are not listed on the SGX. I see the trend
for this increasing. Using ver 2.1, I sort the First Date in descending from latest to
oldest to show the new issues listed in the past two weeks. As you can see most, if
not all, were structures warrants.

As of yesterday, there are a totla of 1213 issues - 733 stocks/others and 480
warrants.

(Posted Image)


To prevent cluttering of the stocks directory, in version 3.0, I will put all the
warrants in a separate directory. This will make many users happy. It makes
their selecting of stocks in their charting application easier.

This will also make it easier for the new product, Analyst's RadarScan.

As promised, this version will have a Warrant Calculator and the Crocker
Price/Volume chart. As promised too, existing users will get version 3 as a free
upgrade. When version 3 is released, there will be a price increase for new users
at that time.

(Posted Image)

This one is live. The SGX subdomian "info.sgx.com", where all the listed company
info and download are, is down at this moment. There don't seems to be any
maintenence annoucement on their website.

This is the new more informative dialog message with improved error-trapping:

(Posted Image)

The new release was delayed. But it will be up by tonight.

On some weekends, the SGX shut down their website for maintenance. The last
major one was on 14/15 Jan.

(Posted Image)


It is very normal that users may not be aware of SGX website maintenance.
During that weekend (and two or three previously) some users had problem
although I had explained the cause during earlier occurrance. I made improvement
here. The error-trapping and message dialog is now more informative.

I simulated a website maintenance and this is the new message dialog. It should
be fool-proof now and no panic that the software stop working.

(Posted Image)



This post has been edited by csk on Feb 3 2006, 12:17 PM

CSK
Well done! Really a value-for-money proposition.

Some new features have been added in response to feedback and suggestions.
The new version should be released during the CNY holidays once I am finished
with updating the help file.

Some of the upgrades:

The EOD Snapshot is a useful tool judging from feedback. It can already sort any
column either in ascending (eg. biggest losers) or descending (eg. biggest gainers,
most active) order. So it was a pleasant surprise when there was a request for the
ability to to copy the EOD Snapshot for pasting into a spreadsheet. Then I thought
might as well export it to a file too. Since not everyone uses Excel, the common
file format would be a CSV file. That goes into this upgrade.

(Posted Image)


A very requested feature is the ability to download/update only the SGX Indices
either later in the night (after SGX upload their second file with full Indices OHLC)
or the next day. This is in this upgrade. Of course this feature only makes the
historical database "complete". Since the Yahoo! Indices download/update was
already added in ver 2.0.3 to enable analysis of SGX and Regional Indices, users
no longer need to wait till later in the night for the Indices.

(Posted Image)


Some users like see the price data in a table. This too goes into the upgrade.
Notice in the screenshot. with the pop-up menu. users can now access specific
stock's information. This is new in this version.

(Posted Image)


Some users find the white background a bit too bright for their eyes' comfort.
So I add the feature to change the background. In Microsoft's programming
language lingo, this is termed as Ledger Color. Initially I try to make it fully
user's customizable but this slows down the refresh when the requested date
is changed (EOD Snapshot) or the window is being resized. So I opt for the
second method which is to build in 30 choices of Ledger Color to choose from.
No speed penalty with this method.

(Posted Image)


For ease of house-keeping I add in a feature to automatically detect expired
symbols for deleting. I do notice that there are times where a big number of
structured warrants from different issuers expiring on the same day. Yes I don't
just wait for feedback and suggestion. I use the software myself and I know
what will make life easier.

Don't be shocked that the Cosco file is listed as an expired symbol and therefore
to be deleted. This is very correct as users know, via the historical adjustment
feature, that SGX has assigned a new symbol F83 to Cosco after the stock-split.
SGX continue to update the old symbol S21 with Monday's close for the last 3 days.

Oh yes, a user suggested that the Historical Adjustment feature be also accessible
via a button in the Download tab, next to the download button, instead of just via
the pull-down menu. This will remind him to always check for adjustment update.
The button has been added. See the second screenshot above.

(Posted Image)



This post has been edited by csk on Jan 27 2006, 12:05 AM

GOH:

This is what I get from the from SGX website on Elite KSB Hldgs Ltd's Proposed Renounceable Rights Issue (Revised):

"The proposed renounceable non-underwritten rights issue (the “Proposed Rights Issue”) of
up to 41,708,160 new ordinary shares of par value S$0.05 each in the share capital of the
Company (the "Rights Shares") at an issue price of S$0.06 for each Rights Share with up to
10,427,040 free detachable warrants (the "Warrants"), each Warrant carrying the right to
subscribe for one (1) new ordinary share of par value S$0.05 each in the share capital of the
Company (the "New Shares") at an exercise price of S$0.05 for each New Share (the
“Exercise Price”), on the basis of one (1) Rights Share for every two (2) existing ordinary
shares of par value S$0.05 each in the share capital of the Company (the “Shares”) held by
shareholders of the Company (the “Shareholders”) as at a time and date to be determined
by the Directors for the purpose of determining the Shareholders' entitlements under the
Proposed Rights Issue (the "Books Closure Date"), and one (1) Warrant for every four (4)
Rights Shares successfully applied for."

Extract of Facts:
A. 1 right for every 2 shares, exercise price $0.06
B. 1 warrant for every 4 rights, exercise price $0.05

Fact B is equivalent to:
B1. 1 warrant for every 8 shares, exercise price $0.05

Here is the computation, assuming share price is at $0.10 just before ex:

a. 8 lots of KSB before ex is 8000 x $0.10 = $800

b. After ex, it become 8 lots of KSB plus 4 lot of rights plus 1 lot warrant. Each rights can be converted at $0.06 for 1 share, total cost of 4 lots of rights conversion = 4000 x $0.06 = $240; while each warrant can be converted at $0.05 for 1 share, total cost of warrant conversion = 1000 x $0.05 = $50. (Time value of money ignored for easy calculation)

c. Total out of pocket cost = $800+$240+$50 = $1090 and we have 13 lots of share at the end of day.

d. The ex-right price of KSB will be $1090/13000 shares = $0.0838

e. The right, before conversion, has a theoretical value of $0.0838-$0.06=$0.023

The split ratio is 1000 to 838.

I think you get it wrong. That $0.06 you pay is for you to converted to a share. It is the exercise price. The right, before exercise, should be trading at $0.023. This is the theoretical value of right and the right holder if do not wish to folk out more money, should sell the right at this price. Buyer of this right pay $0.023 to seller, then pay additional $0.06 to the company to convert into a share. Buyer's total cost = $0.023 + $0.06 = $0.0838, which is the same as the share price after ex.

My data adjustment as handled by my data vendor (Econ) and I don't do it myself. I think they adjust for right but not dividend. (But for special dividend that give out substantial amount of money, I think they did adjust).

Hi HC, thanks for the explaination.

Btw, do you adjust the share price for special dividend?

In the case of Elite KSB, they announce
One (1) Rights Share for every Two (2) existing ordinary shares held , and One (1) Warrant for every Four (4) Rights Shares successfully applied
Rights share issued price at $0.06
Warrant exercised price at $0.05
Let say the price before ex-date is $0.10 for 1 ordinary shares.

a. 8 lots of KSB before ex is 8000 x $0.10 = $800

b. After ex, it become 8 lots of KSB plus 4 lot of rights. This rights can be converted into KSB at $0.05 for 1 share, at additional fee of $50

c. Total out of pocket cost = $800+$50 = $850 and we have 9 lots of KSB

d. The ex-right price of KSB will be $850/9000 shares = $0.0944

e. The right, before conversion, has a theoretical value of $0.0944-$0.05=$0.0444

The split ratio is 1000 to 944. Is that right?
Also, in this case the theoretical value of the rights price is lower than $0.06. That means we are at a losing end???

Hope to hear from you and other experienced friend. Thanks.

GOH:

Let get the fact straight:

1. Stock A is trading at $0.50 just before ex-right
2. Right Issue is 1 for 5. Right can be exercised at $0.40 per share (I change this as I suspect your $0.06 is the price of right, not the exercise price)

This how ex-right price is determined:

a. 5 lots of stock A before ex is 5000 x $0.50 = $2500

b. After ex, it become 5 lots of stock A plus 1 lot of right. This right can be converted into stock A at $0.40 per share, or an additional fee of $400

c. Total out of pocket cost = $2500+$400 = $2900 and we have 6 lots of stock A

d. The ex-right price of stock A will be $2900/6000 shares = $0.483

e. The right, before conversion, has a theoretical value of $0.483-$0.40=$0.083

Since you posted this in the data thread, I presume you are also concern with the data adjustment. This is how I look at it:

$0.50 and $0.483 ex-right is assuming no price movement. So the ex-right basically is a type of split that make stock A's price changes from $0.500 to $0.483, or a split ratio of 500 to 483.

Do that in Metastock Downloader and the price and volume will be adjusted automatically.

Hope this help.

Hi,

Can someone advise how do we do the share price
adjustment for stock rights issue.

e.g.
for Stock "A" trading at $0.50 before rights issue.
If the rights issue is 1 rights for 5 existing shares
at $0.06, what is the price after the ex-date of
rights issue?

many thanks.

I have shot off from my mouth too early. There was in fact no error from Yahoo!
regarding the All-Sing HLRS index. I found out only yesterday when I looked at
the chart with data from SGX. I then double-checked with the SGX source. The
source from SGX did show that gap since Monday.

So the right thing for me to do is to admit my mistake here, to correct the wrong
picture conveyed to you early this week.

I went to find out why is this so. Raffles Hldgs is a component stock of this index. I
did an historical data adjustment for Raflles Holdgs ex-date Monday for special
dividend, the day the gap appeared. I also remembered that both Raffles Hldgs
and All-Sing HLRS spotted a similar gap up few months ago and, if the data was
not adjusted for special dividend, a gap down on Monday too. I loaded their data
and yes I found the problem.

The sad thing is I also have to come to the conclusion that if an index can be
affected in this way then the index cannot be a reliable barometer. Whatever the
methodology for the index calculation, either someone forgot to make adjustment
to the formula or the methodology is flawed.

(Posted Image)

The Yahoo! worldwide indices feature is done and released as ver 2.0.3. I hope
that everything goes well. Can't afford any further delay and has to work around
Yahoo! problem.

Yahoo! SGX indices historical data are not updated timely. So a gap may be seen
depending on what part of the day. But if you update the data every evening then
you won't have the gap problem. This is no fault of the program. The issue is the
timelines of Yahoo! updating their historical database.

SGX is about to start trading soon. So Yahoo! current indices will reflect today's
prices from 9:00am (with a delay) onwards. Yesterday prices will not be updated
into their historical database until some time in the afternoon. Some time another
day later. Therefore the gap.

US and some Europe indices don't seem to have such issue. Perhaps those parts
of the world contribute to Yahoo! revenue and too important and costly to neglect.
Can't blame them. They have to survive.

Ver 2.0.3 also use the "hybrid codes" for updating. "Hybrid" because it uses part
of my fast code routine and part of the standard generic code routine. My fast
code routine does the job faster than the generic code routine by about 60 pct.
But because SGX sometimes add a character into their file that breaks my fast
code routine dead, I implemented the standard generic code routine that won't
break but runs slower. This "hybrid code" now speed up the updating process by
about 23 pct. I am trying to see how else I can improve it.

Haven't yet gotten the time to send out circular to registered users to download
the new version and the updated database that include the Yahoo! worldwide
indices history. Maybe after I go enjoy a bowl of peanut porridge for breakfast.

Add-on: Comes complete with Yahoo! price error on All-Sing Hotels.
(Posted Image)


(Posted Image)


(Posted Image)



This post has been edited by csk on Nov 16 2005, 08:52 AM

I kept getting this error. I kept checking my codes. I kept thinking I was doing
something wrong... until I fired up IE and checked the page. If only I had truested
my IE more than Yahoo!, I would not have wasted so much time. The screenshot
below shows the problem. The "latest" prices are different because the captures
were done at different time. But look at the ridiculous error from Yahoo!!! Look at
what is causing that gap with their current day prices! Why can't they synch it with
their historical which do not have this BIG gap error?

This much requested feature including Worldwide stock indices should be released
tonight as ver 2.0.5. (well including the error from Yahoo since this should not be
a show stopper and since the historical data can easily be corrected later by a
website uploaded file.)

Meantime, before I do, I will have to prevent the database from being updated
with erroneous weekend prices like what Yahoo! provided this morning with the
SSEC index - erroneous OHLC for 13Nov2005 yesterday a Sunday.

Philippines index is out because Yahoo! don't provide it at all. Thai SETI is also out
because Yahoo! don't provide the history althought they provide current day prices.
But without the history, it will not do.

I am also looking at reimplemeting my fast updating codes (after properly
handling SGX addition of a character in their file, infrequent but it happened) in
place of the generic errorfree but slower codes. The updating can be about 40pct
faster.


(Posted Image)



This post has been edited by csk on Nov 14 2005, 11:33 AM

See what I mean now? Same blast, different time.

Maybe I should really consider making the files available but with really bad prices
for good symbols and let serious damage be done, if this is a effective way to stop
this.

Or maybe a better idea - I put those files there but not data files but viruses
and worms and let him have them if this is another effective way to stop it. But
first, put a warning on the webpage - "These are viruses and worms. DO NOT
DOWNLOAD!" And if still do, hey, I already warn you not to do it!

(Posted Image)



This post has been edited by csk on Nov 9 2005, 10:08 PM

Not related to this thread, but share it here anyway.

It looks like a sort of Internet worm. I see a lot of this the past two weeks.
Fortunately, my web-hoster is good.

A few months ago, my website was attacked daily by a worm from an IP address
belonging to a local ISP customer. The website was trying to fake our non-existing
email address to send emails. No luck for them. What ever precaution my web-hoster
has put in place, it was simply no go for the worm. My first two reports to the ISP
was replied that problem was solved. But the problem continued daily. My third
report to them was copied to ida.gov.sg. Problem solved immediately.

Fortunately, our email server is secured and cannot be used to send emails to
external addresses without... er... well... er... cannot say lah. Must not reveal
the safety feature.

(Posted Image)



This post has been edited by csk on Nov 9 2005, 12:24 PM

There have been many instances of abuse where my server was blasted with
repeated requests for the EOD files even for weekend dates. Sometimes these
requests were repeatedly made in some sort of a loop. So this rule out manual
requests.

Analysing the error logs, shows that such requests were not made through
browser (which would have shown up) but from a software written for this very
purpose. My helpful intention has been abused and this is also one of the reason
why I wanted to stop the free download. This is a small reason but a very irritating
one. The main reason is of course to free back my time every evening.

What is today's date? 9Nov2005. See the dates of files that are being requested?


(Posted Image)




This post has been edited by csk on Nov 9 2005, 11:56 AM

Some part of the SGX website network were not accessible intermittently on
Saturday and early Sunday. I had not expected this but then I should have
thought of this because it is normal that website owners do maintenance work
during weekend.

However if I had or were aware I may still not know what errors to trap in the
return file when a download was attempted since I have not seen such a return
file error before. The program does handle other more common file download
errors but not this particular error. You see even if you can't download the
intended file, you will still download a html file telling you the webpage is not
accessible. This is how the web HTTP protocol works

Fortunately when I received error reports on Saturday night, I tried a download
and true enough I was getting the same problem. So I sat through yesterday
(Sunday) thinking through and testing codes to handle this possibility..

Pretty rush but I put in the extra error checking codes and uploaded ver 2.0.1 last
night.

I may have to add in anoption of not automatically effecting company names
and therefore allowing custom naming by users. The SGX uses all capital letters
and a feedback question is can custom names (non capital letters and shorter too)
be used. Possible but only if auto name change is turned off. Users who prefer not
to will then have to manually change it. Recent example is Hua Kok to Abterra.
Will improve the speed of updating too since not have to do any checking but
there will be problem for the Warrant Calculator built-in "date intelligence".



This post has been edited by csk on Nov 7 2005, 12:04 PM

Just did a check on Yahoo! historical data on SGX indices for last week's hoildays.
Looks okay, no zeros prices. They might have solve the problem of historical data
judging from there being no zero prices too on National Day which was a weekday
back in August. The problem now is how to handle the Yahoo! current price error
in order not to corrupt the updated history on the database in the local drive.

(Posted Image)



This post has been edited by csk on Nov 7 2005, 11:38 AM

It is very bad today with Yahoo!

Look at what they are providing. Their current prices for SGX indices show
yesterday date but look carefully at the Open, High, Low and Close. There are all
completely wrong. What this means is that if you have program codes to update
current day prices then what happen is that you update all these wrong prices,
over-writing yesterday's good price with today's big bad wolf prices. Your codes
will not be updating current date but overwriting yesterday's prices. Woosh! Gone!

Totally ridiculous!!!

(Posted Image)




This post has been edited by csk on Nov 3 2005, 10:26 AM

JKSE is holiday today. Like KLSE yesterday, the prices for current session
correctly reflect those of the previous day, the last business day. So there is not
likely to be any problem downloading its current prices from Yahoo! since it does
not show errorneous prices during holiday. The historical data for KLSE and JKSE
are also okay, at least at this point in time. Can't tell whether it will be same on
another day at another time.

Sadly all this out of the window or into the thrash bins when it comes to Yahoo!
SGX data. I have already trim my hair like durian so that I cannot pull anymore.
Don't know what else I can do. Maybe go find David Copperfield perhaps.


(Posted Image)


(Posted Image)


(Posted Image)



This post has been edited by csk on Nov 2 2005, 04:12 PM

Yes, exactly as described.

I am wondering how can I write the codes that can check and determine that
20Jan2004 was NOT a holiday? Include a holiday file? Not practical because that
would mean updating all the holidays all over the world in that file and to be able
to determine which country the symbols are from. Then there is another problem
where a country may be in holiday but the stock is still traded. Possible? Definitely
yes.

How about the current day as shown in my earlier post on SGX indices? Tell users
they must never update from Yahoo! whenever there is a SGX holiday otherwise
their data files will be corrupted? Not practical.

Anyway, it is still early to see what Yahoo! will update into their historical database.
Have to wait a few more days.

(Posted Image)

Yahoo's data is not very clean. That apply even to their US stock data. I had been using MLDownloader for a few years, and I had seen this so often that I have given up system testing on US data.

The problem can be in any of the following forms: zero opening price, zero HLCO, missing day(s), wrong data (eg: stock that usually traded at 10, and you see suddenly somewhere in between the historical data, a single day traded at 40. Upon check with online chart from other source, confirm it is wrong data), inconsistant date format.

They seems to do update/patches every now and then, but the patch is not perfect and the update is seemingly random. The problem can be in a single stock, or a few stock, or across the whole exchange. And if downloading is during the trading hour, partial data of the current day is also downloaded as EOD.

In MLDownloader, I am able to specified a period of historical data to download. So if a single day or period that is problematic I can try later to dowload again to overwrite. If I am downloading during (US) trading hours, I has to specified a date that is 1 day before so that my EOD data is correct.

Hope that the above experience will help those who wish to use Yahoo's data.

This post has been edited by Hc on Nov 2 2005, 12:49 PM

So as I has expected, there is a problem with getting Yahoo! data for current day
when it is a holiday - Deepavali.

Look at the screen capture below. This is a snap shot of current day index prices
from Yahoo! See how the High, Low and Close were filled in with yesterday's
closes today's open were filled in with yesterday's open? Although I can write
codes to check but it is not efficient and proned to delay and error.

KLSE is also a Deepavali holiday today. And that's how the prices should be
presented by Yahoo! Yesterday's prices with yesterday's date. There is no
problem with this. Immediately you know you are viewing yesterday's prices. If
you update them you are updating to yesterday's date. This is how it should be.
Yet all from Yahoo! and they are not consistent with what they are doing.
Tomorrow, will have to check what they put into their SGX indices database. Will
they be zeros "0" ???

This remind me of personal experience. This is very typical of foreign talent and
yet our Govt keep singing praises of them foreign talents. I have worked with
foreign talents, sitting next to them in Singapore, communicating with those in
their foreignly talented lands. They are just empty fronts, completely hollow
inside. It is all just personnel glooming and marketing with absolutely no single
drop of talent at all. Ask them some questions you need in the course of work and
expect them to know, they become all defensive, throw smoke and get abusive.
Never able to provide the answers but go all out to get you fired because you are
a nuisance to them.

That's how a once big time financial info service provider company went bust.
Killed by them foreign talents.



(Posted Image)




This post has been edited by csk on Nov 1 2005, 07:53 PM

It's been a hectic 2 weeks since the annoucement. Almost immediately, I recieved
requests for the ability to update Worldwide stock indices from Yahoo! While this
was in the pipeline, those who inquired wanted this in the release. So I tried to
rush the codes but something somewhere acted up.

I was still having intermittent problem with SingNet broadband and some data I
got from Yahoo! were completely out. I don't know where the problem lies but it
did show up a very important point - that codes need to be thoroughly tested to
iron out any kinks. So in the end I held back this feature from the software and
suffered a one day delay due to unexpected installation setup problem that
cropped up on Wednesday. This could have been detected earlier if not for the
attempted rush on the above-mentioned feature.

Now that it is officially released, I am or course back to punching codes, to make
the requested funstionality in the next feature release.

In the meantime, I have to see what develops this coming week with Yahoo! when
Hari Raya and Deepavali come around. See the Yahoo! All-Sing chart below. See
the many zeros "0" that concide with Singapore holidays? Another reason why I
decided not to include the feature in 2.0.0! I don't want to risk it with an
unnecessary bad image just one week after its release even though no fault of it.


(Posted Image)




This post has been edited by csk on Oct 30 2005, 05:01 PM

Ever since Technical Analysis & Research announced that they are stopping to provide free SGX EOD data, but those who need it has to go through their proprietary software (cost $88, one time), I was asked a few times where to get FREE data, so I posted this comment here.

First of all, I don't know of any free and good SGX EOD data data provider.

That said, there are still free data available: from Yahoo and SGX. I don't use them, so I don't know what is the URL and how to convert into metastock format. But from what I understand, the conversion is not as simple as what you used to see in Technical Analysis & Research download file. If you MUST get free data, then try figure it as I am unable to help.

Secondly, I wish to point out that Technical Analysis & Research's $88 one time software fee and no monthly subscription is excellent deal. I am sure that you time and effort to do downloading, error checking and conversion of data daily is worth more than that $88. The time and effort save, can be better used to do analysis, or whatever things you like.

When I first decided to take on paid data more than 10 years ago, I paid $20 a month (on top of a one time fee $99 for starter pack which consist of historical data + downloading software + 3 months trial) and I traded very small and very infrequently. My rational is: if this data can give me a better entry or exit by just one cent, any 2 lots trade each month already cover the cost.

As for those who are still undecided to spend that $88 since you just started TA, think of it this way: That $88 give you access to the live data of the whole exchange, which you can do verification / testing of whatever you have learned. I beleive any trading mistake you avoided by doing so will save you more than $88.

In the worse case, you find that TA is not you cup of tea and that software is useless, you downside is limited to $88. (We are talking about trading/investment here, so we should always know what is our downside.). Is that too great a pain?

This post has been edited by Hc on Oct 29 2005, 07:15 PM

Thanks Hc for making the annoucement.

There will be a lot of checking to do the next few days before the software is made available for download by those who buy the license - a one-time S$84.50 or US$50.

A common question asked and one which was also commonly asked 6 years ago in 1999 was what if the SGX stop making the file available or they start to levy a fee? My answer then is still the same as my answer now. There is competitive pressure on exchanges worldwide to make the daily prices available to attract business to the exchange. This is also to take advantage of the functionality of the Internet to stay ahead of their competition. But still will they stop it or start charging? There is always the risk they may. I cannot say they won't because I am not them.

But at $84.50 or at S$88.00 previously, let's look at it from another angle. How many months of data service complete with historical adjustment for corporate action can one get from subscription data vendors? The subscription data vendors charge about S$20 a month. Some charge more. Some make you pay for the historical data. How much for the historical data if they make you pay? S$50, S$60 or $80 depend who you go to.

But even without paying for the historical data, after 4 months, that's it. S$80 gone. If you subscribe for a year it will be S$240. Those who bought the first version of the Analyst's DataServer, they have used it for 6 years already. That is 72 months (6years x 12 months). They would have paid S$1,440 for the data if they had gone to a data vendor. But they took the risk and paid only $88. Yes they took the risk and because of that, by now, they have saved themselves S$1,352 ($1,440 - $88) and they will continue to save the longer they use the software. To them, the software has long paid for itself. This is the angle I prefer people to see. This is the real truly beneficial angle.

I still remember one of them told me that to him the $88.00 is just for the historical data, the software is a bonus for him to get free data every day as long as they are available. This was a very nice thing to hear. Of course for me I don't sell the data. I sell the software and the historical data comes free. From then on the data comes free courtesy of the SGX.

I still do the historical adjustment work. And now a major software upgrade. For those who have already saved S$1,352 they will get this upgrade free. You don't get this type of service from free data source, except maybe from me the last 6 years.

The market for this software is very small. There are not many who will pay even a one-time fee even when the savings in the long run is significant. They will try to get the data free somewhere. I don't expect to sell a lot of copies of this software.
I have even provided a link on where to get the SGX file. I know a number will try to DIY. But after a while when they find they can't keep up with the historical adjustment they will give up.

I just want to highlight the angle I prefer people to see, the real truly beneficial angle.



This post has been edited by csk on Oct 19 2005, 06:53 PM

ahh... csk = Mr Cho Sing Kum ? :rolleyes:

I'm still reading more about the changes. Thanks HC and csk :D

.

Technical Analysis & Research had made changes to the free download of SGX EOD data from 2005-10-31: you have to do it through their updated software called Analyst's DataServer. This software costs one time fee of US$50 and does conversion into Metastock format automatically.

See http://www.technical-analysis.com/prodSGX.html for more details.

Since this data source now come with market breadth data (like TRIN among others) which is hard to come by, I think it is worth your support to Technical Analysis & Research.

The software is about complete, left the checking for and downloading of adjusted
histories to be done. The program has links to SGX webpages for information.
Some of the links are approved by the SGX today while pending a few more to be known later.

If everything goes as planned, this will again be the first of its kind repeating the first of its kind feat of its previous incarnation.



This post has been edited by csk on Oct 12 2005, 01:30 AM

Day in day out you have been bombarded by daily listings of stocks with highest
volume traded and stocks with highest value traded for the day. You see these
lists everywhere; on SGX website, on Internet sites, from brokers or their
websites and the worst part from the high priced financial information and data
vendors that professionals pay a bomb every month for their information.

But have you really taken a look at these lists and see whether they are useful or
helpful to you? I will show you serious errors these people are making and yes
even the big time financial information companies.

Fig 1 to Fig 3 below are screen shots from the new DataServer for SGX ver2005.

Fig 1 shows the stocks sorted by biggest volume in descending order. It is similar
to what you would see everywhere. But! there is a difference here. Those using
services from the big time financial information companies may see what you see
here in the screen shot. They will see that the third biggest volume stock, L & M,
has only a meagre SGD 117,560 worth of stocks traded. How's that for realization.
If you have seen this earlier would you still its third place worthy?


(Posted Image)


Fig 2. This is where the big mistake is being made every single day and this is the
list that can mislead people. The stocks are sorted by value of the stocks traded.
These values are disseminated by SGX to data vendors. All data vendors and
Internet financial websites, I believe I am still right on this but stand to be
corrected, all of them simply regurgitate these values in their information
services, including the big time financial information companies.


(Posted Image)


The SGX is not wrong in trasmitting these values. They are useful. It is up to the
data vendors who receive this information how they should massage them before
showing them. And all of them, I stand to be corrected on this, simply do the easy
thing - just show you what they get from the SGX feed. And when you ask them
they tell you it is from the SGX. Worst some of the data vendors even provide you
the sort function, like that of the DataServer shown here, without converting them
all to a common base that makes such sorting useful. This is where the software
shines. Again, a Singaporean has done what foreign talents could not do.

Fig 3 shows the correct way to sort the value information. They have to be
converted to, what else, SGD! So whereas both Jardine's were not in most of the
others' wrong lists, they are rightfully in the correct list!!!


(Posted Image)


This is a sample of the value add I want to put into the product. You notice the
date-picker in the software which means that of course that past days lists can be
retrieved. Where else can you get this? I tell you, no where else.

Fig 4 is one for the day after 911. You definitely cannot find this anywhere else
now.


(Posted Image)




This post has been edited by csk on Sep 28 2005, 11:22 PM

The Quote Window actually comes from the Raffles Place project. Here is shown a
test run with US indices and stocks refreshing at 1 min interval.


(Posted Image)


I made sure that intermittent Internet connection issue did not hang the program.
The quotes on display is program to "gray-out" when updates failed to be
retirieved. The log keeps track of connection problem. Yahoo server usually
respond quite fast. I am experiencing Internet connectivity problem yesterday
and today with SingNet and is a good time to test out the program codes.

(Posted Image)



This post has been edited by csk on Oct 12 2005, 01:24 AM

If you are using Microsoft AntiSpyware, you will know this is the menu to check for
updates for new spyware definition.

(Posted Image)


The check for historical data adjustment will be something similar. The adjusted
files will be on the website and the program checks and fetches them. I prefer this
manual process to avoid the wrong impression that the program is automatic
"calling home" with sensitive information. The program then log-in to a FTP site
this is why I prefer the user initiate this manually.

Still thinking whether and where to put this and whether to do it now. The EOD
snapshot will look something like this except that this one can be configured to
update every few minutes with Yahoo! data. Symbol list can be customized.


(Posted Image)



This post has been edited by csk on Sep 27 2005, 12:26 AM

Putting the parts together into the shell. This is how the final product will look like.
Getting near to the reincarnation.

(Posted Image)


The EOD snapshot is a quote table where you can sort by Gainers, Losers, Volume, etc.

The downloader goes here:

(Posted Image)





This post has been edited by csk on Sep 27 2005, 12:08 AM

By Golden in the CADI, Cumulative Advance and Decline Index thread:

I have plans to include the CADI and CVI numbers and probably also the TRIN as
seperate files in the Close field. This way plotting the files as Line chart of the
Close will produce the respective indicators. There will not be the need to have to
construct the indicators from seperate files like what the monthly subscription data
vendors want you to do. If you read the previous posts in this and the CADI
threads, you may know that I do not intend to do this in the present daily ASCII
file but to let the new Downloader do the work.

I am also thinking of how I can do the New High and New Low numbers since this
requires checking the price data of all the stocks. I want to do this as efficiently
as possible. I am going through books and magazines on what other info traders
in US stocks have at their disposal to work with to see whether I can also provide
the same here for SGX stocks through the new Downloader.

I wrote the first ever commercial downloader a few years ago that download the
SGX file and (1) post them to an ASCII database and (2) create an ASCII file for
conversion by the MetaStock Downloader. I also provided the adjusted history
files. Since then, similar software from other companies have been released.

I don't want the new Downloader to just do the same plain old thing but to also
provide new info and features. Nowadays, many people can do downloaders but
unless they have the exposure and experience they won't be able to come out
with new features that can help users to improve their readiness. Probably I might
be the same one to set the example again and then others would follow.

One such area is structured warrants. These are getting quite popular but there
are many who still don't quite know the basics. So I intend to tag features onto the
new Downloader. Warrants, like options, can be cheap or expensive but not in
absolute money terms but with what implied volatility they are trading with. If you
but a call warrant trading at high implied volatility and even if you get the direction
right, the mother share goes up, you still may not profit and even lose when th
implied drop at the same time.

The new Downloader is not a freeware. I foresee regular revisions in response to
whatever changes made by SGX. I am thinking of what info I can include beside
the normal price data. If it is to be a leader setting product then it has to be good.
Maybe not in its immediate first reincarnation but at least quickly evolve.

I thank you for your interest in the data. This is one of the many request I have
had for the Adv Dec data and I want to make them availlable.

Back to my books and magazine now.

I put the automatic multiple days downloading/updating feature of the new
Downloader to the test today. SGX still have files back to July and I specifically
wanted to test the automatic multiple days download over a weekday holiday.
9 Aug was a weekday holiday so I did a download and update from 5 Aug right
through to 21 Sep, a total of 34 market days.

Total running time was nearly 2 hours giving an average of 3.5 min per day. This
include undating the MetaStock database and parsing and updating the AdvDec
issues and volume numbers.

It passed the test without any hiccup. I also tested the downloading process
behind a corporate firewall and again no problem.

I will be adding a very simple chart just to plot the price and volume, nothing
fanciful for the moment but helpful for visual flipchart scan. This chart function will
be a strip-off from the Turtle program. There should be the ability to list stocks
according to sector indices which I think will be useful.

And further down the development, a warrant calculator with option-type chart.

When I checked today at 5:50pm, the SGX file was already available.

For the past 3 years or so every evening I have been doing the MetaStock
Downloader compatible ASCII file on requests from SI forumners when SI
decided to stop the daily download around Aug 2002. I have wanted to reclaim my
time every evening so that I can be free to do what I want wherever I need to be.

I am not a data vendor and I really want to get out of this. So perhaps re-releasing
the new version of the downloader may be the solution.

The only snag is SGX does not provide the open, high and low in their first upload.
I cannot understand why. Their second upload around 10:30pm will have them. I
really cannot understand why.

(Posted Image)



This post has been edited by csk on Sep 21 2005, 06:26 PM

I've been changing the downloader software a bit to include features that were
requested. I am now testing it and using it.

Some additional new ideas sprang up in my mind today while replying to Lion on
Penguin warrants at this link:
http://s7.invisionfree.com/ChartistsUnited...dpost&p=2778225

Maybe I can include new ideas to include/produce information similar to those
available in the US on US market but info that are sorely just plain not available
here, to include into the software turning it into a toolkit and re-release it back to
the market.

There have been request for the old downloader I wrote few years ago and which
I have already sunset. If I were to re-release the new downloader with new
ideas and can produce additional new info/statistics, it would require regular
updates/upgrades. I don't know whether there will be enough demand to make it
feasible.

And then I also have a warrant caluculator with options-like chart in mind. Merge
them into one?

(Posted Image)




This post has been edited by csk on Sep 21 2005, 05:31 PM

For quite some time now, it has been quite irritating that the SGX has gone
bonkers with their symbology. Stock whose names begin with S have been
assigned symbols beginning with T, then U, then V. The same problem is
happening with some other letters. The problem is made worst by the many
listings of structured warrants that use up symbols very quickly and then expire
after a few months. This makes SGX's lack of planning so glaringly awful in the
International arena. Why can't they just one more number to make it 999 per letter
instead of keeping to 99 that is now obviously insufficient.

I am finding it increassing difficult to locate some stocks since I use "traverse
destination folders". I am now beginning to use my own Downloader for SGX data
and am now deciding whether to still traverse destination folders but instead of
MetaStock Downloader's method of following symbols I can make my own baby to
traverse follwoing stock names. Therefore the two new stock that are listed today
with names beginning with "S" will still be posted to the "S" folders instead of
the "V" folder.

Currently I dump all the stocks into one folder and the 16 indices into another
folder.

While I am designing my own downloader, I am also thinking of parsing the file for
Total-Advance-Decline-Unchanged numbers and post them to a file with the Total
Issues going into the Open field, the Advance into the High field, the Decline into
the Low field and the Unchanged into the Close field, just like in CompuTrac.
Indicators can then be created to do the neccesssary calculation.

Naturally, Advancing Vol and Declining Vol are also possible and this can go into
the Volume and OpenInt fields.


(Posted Image)




This post has been edited by csk on Sep 2 2005, 08:51 PM

nasal:

1. Please give more details if you are asking for help.

2. MLDownloader is not a free program. You have to pay for it in order to continue using it.

3. If your had already paid up and still not working, check with the developer's site for support.

HC,
whats "access denied!" when trying to download using mldownloader?

You must make sure the tools you use are reliable.

There is obviously a serious error in the chart at the top. The candles
presentation are all wrong. Since the data source is the SGX there should not
be any problem with the data. The problem is with the graphing routine.

(Posted Image)


Then again there is also another problem (See big chart below). One where you
are so obssessed with a particular view that you start putting data in where there
is not so as to to let you see what you want but it is not there. This is illusion.

Remember the test I asked you to do?

(Posted Image)




This post has been edited by csk on Mar 28 2005, 12:24 AM

You are right H and U are the sub-folders, my typo error there.

No I do not have the sub-folders created. But reading what you wrote I knew you
would have this problem. So it is either you put everthing (including indices) into
one folder or you travserse everthing into sub-folders. There is no in-between like
what you are trying to do.

I was looking at my ASCII set of files to see what are the new listings.

Hi HC,

Thanks for your 'dead counter' routine.


Hi CSK,

Thanks. I forgot to mention in my last post of Mar 22 2005, 03:53 PM about new symbols reated. Last night's update created 6 new symbols in 2 new sub-folders - 2 for "H" and 4 for "U". Dunno why your new subfolders are "D" and "U".

Prior to 'meeting' with you guys I had only used MS to manually record something like 20 SGX counters. So MS was more to graphically record EOD prices. So, I was using a ferari to just make runs between home & the grocery stores. Seeing your discussions on your Raffles Place & Turtle Trading System projects I realise I am in a way witnessing some pros designing the next generation super car. Wow!!

rgds

eric

Hi Erichew,

The re-assigning of symbols is just an alternate thing which I did not do so don't
worry about it.

If you have done the conversion yesterday with the "Traverse Destination Folder"
checked, then there will be a total of 6 new symbols that should be created
in sub-folders - 2 for "D" and 4 for "U". These are the 6 new symbols in
yesterday's file. The Donwloader would not be able to find these symbols in your
destination folder and since you checked, they should then created in sub-folders.

On your question to delete or move files, you can do it via the Tools menu.




This post has been edited by csk on Mar 22 2005, 05:32 PM

erichew:

A work around to delete dead conters is posted here:

http://s7.invisionfree.com/ChartistsUnited...p?showtopic=251

This post has been edited by Hc on May 12 2005, 05:00 PM

Hi Hc,

There are two products under development. The one that started first is
codenamed Raffles Place. It is intended to be a general-purpose (GP) type TA
software much like SuperChart EOD or MetaStock EOD. The early plan was to
follow it up with a realtime version.

However, there are already a number of competitive GP software around. And
there are also "plastic wrap hanging on hooks" version of Metastock in
neighbourhood shops so therefore the new plan for the second product.

This second product is codenamed Raffles Place Turtle System. It breaks from GP
type and instead focus on the complete trading approach on system testing and
trading on portfolios of instruments. Initially the first trading system is the Turtle
Trading System. It retain the Raffles Place codename because it uses the same
graphic engine as the first product.

Unlike most TA software that use 3rd-party graphic engine, this graphic engine is
self-developed by me. I did this after evaluating graphic engines including the one
that TradeStation uses and find that there can be limitation and the possibility that
further development is at the mercy of 3rd-party developers.

The second product has overtaken development of the first. Firstly because of the
competition or rather lack of it and secondly, I am trading the system so I know
what I want and need. Perhaps another 2 or 3 months should see the release.
Then I will work on adding more trading systems to improve the product. The most
important thing about the software is that it will evaluate a portfolio of instruments
collectively and not just simply on only one instrument at a time which is what
most people are doing with existing software.

This way there is no need to make a switch. Keep existing software for GP work
and Raffles Place Turtle System for portfolio evaluation and trading. When it is
ready I might do Turtle Trading Seminars so that it can be a complete learning
experience. I might have a date in KL but I don't know whether the software will
be ready by then.

I will certainly inform you of its release.

Many thanks to CSK :) for your 'tutorial'. I never saw the need for a dual screen set-up. But your instructns for setup seems so easy, & I do hv an extra monitor from time to time. I might try it.

Last night I tried the DownLoader with all data grouped into 2 subfolders i.e. Index and Stocks. With the "Traverse Destination Folder" checked, the update for 21 March 2005 worked, apparently without any problem. I tested a number of Indices & Stocks & they were OK. As this option worked I did not see the need to try the update with "Traverse Destination Folder" unchecked. BTW, after consolidatg into 2 subfolders I deleted the rest, before the update run.

Not sure I understand your quote

. To me the dumping of all stock into a single folder appeared very mechanical (no checking of any list required). Perhaps I may be missing something :huh:

One thing bugs me tho'. When a warrant say expires & if you want to either delete the old data or moved them into a 'retired' folder, how do you do it?

I am sounding more like database worker than a stock trader using TA. Fact is I am no trader. Yes I invest a little, that's all. But am fascinated with what Metastock can do, in the hands of experts like you & Lion &...etc With time I might be a convert :)

Thanks again

Rgds

eric

CSK please let us know when your Raffle's Place is ready.

There is advantage to have all the stocks in one directory. You don't have to
always browse to other sub-folders whenever you want to change the stock to
chart and its symbol happens to begin with a different letter. Also you can get
around the problem of having "S" stocks in "T" or even "U" sub-folders and "C"
stocks in "D" sub-folders. Since the symbols are assigned by SGX and we cannot
do anything about it, it is becoming a real mess as days goes by and new issues
get listed. Unless we re-assign our own symbols which then will become a tedious
chore with daily checking against the list required.

This software reads the present MetaStock format, meaning it reads beyond the
first 255 files. It also can multi-launch, meaning more than one instance of it can
be launched on one computer at the same time. Just like you can launch multiple
instances of Internet Explorer or Notepad or Wordpad. Like this, it is good for
people who has dual or more monitors. So if you have a four-monitor PC then you
can run launch four instances of it, drag one into each screen and maximise them.

I show here two instances of it running, one with the files listed at the begnining
and the other with the files listed at the end of the same folder.

ps Do you have a spare video card and a spare monitor lying around and
you don't know what to do with them? Well you can turn your PC into dual-screen!

Windows XP allows you to do this without you buying those expensive dual-head
video card. Plug that card into a suitable slot, plug in that monitor, install a WinXP
driver for that card.

Then right click on the desktop, choose properties in the pop-up menu and in the
settings tab go claim your prize.


(Posted Image)



This post has been edited by csk on Mar 22 2005, 01:55 PM

Hi CSK,

Thanks for your quick response. You have answered my Qs. I was concerned abt new IPOs i.e. how Downloader wld handle them if "Traverse Destination Folder" is marked check or uncheck. I will test as you suggest. :)

So, the cap for Metastock is 2000...shd be enuf for SGX.

rgds

eric

Whether leaving the "Traverse Destination Folder" uncheck will cause error
with daily update is difficult to answer.

Software features may change from version to version. What may work in one
version may not work the same way in the next version. There are many versions
of The Downloader so the only way to really find out is to run this test on all
versions of The Downloader.

If you want to put all the 900-plus symbols in one destination folder then the safest
thing and the most consistent is to uncheck it. You don't know when there
will be new IPO. You also don't know when there will be new structured warrants.
If you check then all such new symbols will definitely go into new A-Z
sub-folders. This part is consistent through all versions of The Downloader.

The MASTER holds records for 255 files - CompuTrac (and old MetaStock) format.
The XMASTER holds records for 2000 files - new (current) MetaStock format.

From Help file:
Error #1107. "XMASTER: Number of stocks invalid"
Your data folder must have between 1 and 2000 securities.



This post has been edited by csk on Mar 21 2005, 09:53 AM

In fact, CSK has answered your question previously

Make sure to uncheck "Traverse destination folders". All the updates will then go into the destination folder and will not traverse into A-Z sub-folders.

Only MetaStock will read beyond the 255 files.

Hi csk,

I am new here. Read this thread. I had downloaded the historical & daily data from TAR, and like new2005 I prefer to have all counters in the same folder, altho I have yet to effect it.

Taking up from your post of Mar 16 2005, 01:06 AM, if I were to use The Downloader's Tools|Copy... menu to copy all to a single folder and delete the now empty folders, can I leave the "Traverse destination folders" checked without causing any error with the next daily update?

Also,.....does Metastock have a cap on the number of counters you place in a single folder?

Thanks

erichew

First, you have to copy all the historical files from the A-Z sub-folders to the destination directory where you want to store all the files. You can do this from The Downloader's Tools|Copy... menu.

Then when you convert the EOD files everyday, make sure to uncheck "Traverse destination folders". All the updates will then go into the destination folder and will not traverse into A-Z sub-folders.

Keep in mind that once you do this, all other software will not read beyond 255 files since they only recognise the Compu-Trac format from which the Metatock format is derived from. Only MetaStock will read beyond the 255 files. So if you using software other than MetaStock then you should not do this.



This post has been edited by csk on Mar 16 2005, 01:13 AM

hi,
currently the imported data are stored as “A” stocks would stored in the “A” subdirectory.
how can i import and store them all in one folder ?
thanks

High quality SGX end of day data available in MetaStock format:

http://www.premiumdata.net/products/premiumdata/sgx.php

Also available: Futures, Forex, Australian stocks and US stocks data.

Hi CSK, Good News.. I solved the problem, after 'study' your 3.31pm post. Yes, I did converted the Daily datas to wrong directory. Actually, there is another 'Ses' folder inside 'SES' folder! :(

Thanks to Lion and HC too!!

This post has been edited by new2005 on Feb 25 2005, 08:10 PM

Oops!

My post earlier at 3:24 pm local time (2 posts before this) should not be in this thread. It should have been in the New Wave Share Conso, Be careful with pre-opening prices thread since this is what it is about.

I have reposted it there. Sorry.

This post has been edited by csk on Feb 25 2005, 05:06 PM

Hi new2005,

This is a screen shot of my Windows Explorer showing the actual destination and
sub folders where all the data are.

In my case the Destination Folder is "C:\TAR-Data\SES". This is the path I put in
the Downloader's Destination Folder input box. Since I set traverse destination
folders, the data automatically go into the sub-folders "0", "5", "6", "A", "B", "C"
etc etc. which are one level below "C:\TAR-Data\SES".

Hope this is clear.

(Posted Image)



This post has been edited by csk on Feb 25 2005, 03:32 PM

Hi HC,

Thanks for the links.

So once again the issue of bonus do not benefit the shareholders. They were suppose to gain $0.17 in the whole process but the market took it away from them yet again. Back to square one. We will see a price gap on this chart too.

Listed companies should really forget about bonus issues whether it be of shares or warrants. The shareholders never gain from it!!!


This post has been edited by csk on Feb 25 2005, 03:34 PM

Hi new2005,

I do understand you can read both history and the daily updates but since you are reading them as two separate files then they both cannot be in the same sub-directory. Otherwise they would have merge into the respective same files. This is how the downloader conversion works.

Do a test:
Which directory are you reading SIA history from and which directory are you reading SIA daily updates (8Nov2004 and later) from? Load both charts in MetaStock and pay attention to file browser control to detemine which SUB-directory you load the history from and which SUB-directory you load the daily updates from... RIGHT DOWN TO THE VERY LAST "S" sub-directory (for SIA).

From what you describe, I am very sure they are not in the same sub-directory, meaning you did not append the daily updates to the history.

I hope the MetaStock Downloader's documentation was more clear but unfortunately it is not. This is why you notice in the website I do not want to answer this type of questions. Answering all the downloder emails I received would take up too much time when it should rightly be Equis' job.

This post has been edited by csk on Feb 24 2005, 10:10 PM

Hi CSK,

[Assuming that "C:\MetaStockData2\SES" is the directory where the history is unzipped into then this is the directory you have to input as the destination directory.]

I did as what you mentioned above.

[Since you can read the history but not the daily updates after that, it seems you are not updatng the daily files into their correct location.]

I can read both History and Daily Update datas BUT seperately. Means, Historical datas end at 05-11-2004; Daily data start from 08-11-2004. Both are in the same directory(SES).

[Making a guess I think your SIA history is in "C:\MetaStockData2\SES\S" whereas SIA after 5Nov2004 is in your C:\MetaStockData2\S". Is my guess correct?]

The Daily Update datas are converted as "Fxx" dat files.

Do I provided enough informations to help my convertion problem?

Thanks.

Hi new2005,

After you unzip the historical data, make a check on the actual FULL directory path up to but EXCLUDING the sub-directories of "0", "5", "6", "A", "B", "C" .... "Z".

Since your main folder is C:\MetaStockData2, into which folder was the history unzipped to? Is it "C:\MetaStockData2" or is it "C:\MetaStockData2\SES"?

Since you can read the history but not the daily updates after that, it seems you are not updatng the daily files into their correct location.

Assuming that "C:\MetaStockData2\SES" is the directory where the history is unzipped into then this is the directory you have to input as the destination directory.

Making a guess I think your SIA history is in "C:\MetaStockData2\SES\S" whereas SIA after 5Nov2004 is in your C:\MetaStockData2\S". Is my guess correct?



This post has been edited by csk on Feb 24 2005, 09:13 PM

Thanks to HC and Lion. Sorry for late reply. I still can't combine both historical and daily datas.. as what Lion mentioned we need some other steps to do it. I will only use Daily dowload datas(from 01-11-2004 to study markets). :(

Hi new2005,

Refer the attached word document for steps of conversion which I've previously send to those interested in my email contact list.


Rdgs,
Lion

Attached File ( Number of downloads: 115 )
 Conversion_Steps.doc (44 kb)

new2005:

Check your conversion options: Tools/Convert/Options/Destination tag, make sure you have the [Traverse destination folders] ticked when you are doing both the historical and daily data convertion (or both unticked).

This post has been edited by Hc on Feb 20 2005, 11:24 PM

Hello HC, Thanks for answered my post. I did read the PDF help file. But...

Let me explain in this way:

1) I created a folder "MetaStockData2". This is for me to extract Cho's Zip file(historical datas with last date: 05-11-2004). Done! No problem for my MS program to read it(open chart).

2) In order to get up-to-date datas(from 05-11-2004), then I downloaded Cho's "Daily" file. In this step, I have to use MS Downloader to convert 1 file(1 day's file) at a time. I have converted the "Daily" data to my "MetaStockData2" folder too.

The problem: I thought that the historical data is updated. When I am trying to open a chart, I found that "historical data" is not updated with new data. The "Daily" datas can't merge with historical datas.

To clarify, the historical datas are seperated with many folders, A to Z.

The main Question: Which folder should I convert the "Daily" data to? (Because in this single "Daily" data file contained A-Z market's datas).

Hope HC or other can help! Thanks. :(

This post has been edited by new2005 on Feb 20 2005, 10:34 PM

new2005:

See if the following file, by Cho Sin Kum helps:

http://www.technical-analysis.com.sg/downl...oc/Ascii2MS.pdf

This post has been edited by Hc on Feb 20 2005, 04:23 PM

Hi, I am new here, would you please teach me how to convert the "daily" SGX datas so that it can "joint" the "historical" datas (last date was 05-11-2004)? Thanks.

hotmacca1975:

I only use MLDownloader, so not sure about the other s/w listed thereon. I have not try intraday data, but I think MLDownloader is meant for EOD only. For intraday, you have to use some oither s/w (RTQDownloader?).

Hi HC,

From the Trading-tools.com site, can u recommend which is the best currently?
Btw, can the intra-day data be downloaded also?

Thanks for sharing! ;)

Btw, does anyone know abt a course " Financial Engineering" ? thinking of further my study by pursuing it....any comment? <I am really love trading & Technical Analysis>

Hi. I would like to know how to convert the SGX EDO historical data. They are in DOP and not txt format. So far, I able to convert the daily data in metastock.

MLDownloader

I had been using a program called MLDownloader, which I paid for the downloading software one time, and it can pull almost all the stock exchange data from Yahoo, free of charge. I had been using this for US & HK data for quite 3 years, so far so good. The program has pre-selected counter list & symbols so that you can download both the historical and current data straight away. It convert into Metastock format with out user intervention too. Quit useful and user friendly. The software developer is also quite friendly and the response to bugs/problem is very fast, usually within 24 hours. 2 thumbs up to the developer.

The main trouble is, Yahoo's data is not very clean, it contain bad tick, zero open price etc. If you use it for charting, not much of a problem as the bad tick is often quite obvious. I often edit the tick or just discard it. But it cannot use for system testing, as those bad tick will throw your system testing report upside down.

Anther thing you should take note is: Yahoo site can be quite busy at times, so if you are downloading the whole exchange, it can take as long as 2 hours. So it is a concern if you are not on unlimited broadband plan.

The link is here, you can download a full version for trail for 14 days.:

http://www.trading-tools.com

More links I know but don't have any experience:

http://www.paritech.com.sg/sg/products/data/datadirector.asp

http://www.keyquotes.com.sg/keyquotes/PHP/keydata.php

http://www.neuronetsys.com/

Please share your comment as user if you have. Thanks.

Downloadquotes.com

A cheaper source for data can be obtained from downloadquotes.com compared to Econ, is available at

http://www.downloadquotes.com/en/ .

They have many market, EOD and intraday data. I have not tried their SGX data, but I had tried their HK EOD data for 1 year plus and give up, as I am not very satisfied.

Pro:
1. Cheap
2. Can download many times

Con:
1. Data VERY dirty, at lease for HK data, even after complain and they claim that they have rectify but still the same. (Dirty data will affect the ability to display the chart correctly, candlestick formation, indicator calculation, Metastock's system testing, exploration, expert mode).
2. No auto split adjustment
3. Their uptime and and update time is 1 mark below satisfaction. There is a time when HK market is already in the afternoon session and their previous day data is till not up yet.
4. They cap the downloading in term of MB per day. So downloading of Historical data have to span across a few day (daily data has no problem downloading multiple times though).

Those are my experience 2 years ago and I am not sure if it is still the same. Anyone who has recent experience please provide update.

For those who need free SGX EOD data, Cho Sing Kum have it here:

http://www.technical-analysis.com/prodSGX.html

Remember to do some do some charity if you find the data useful.



Updated on 2005-10-17 by HC:

The EOD data download has to go through their software from 2005-10-31. See:

http://www.technical-analysis.com/prodSGX.html

This post has been edited by Hc on Oct 18 2005, 12:02 AM

Econ Management & Investments Pte Ltd
Tel: 6538 8913 Fax: 6538 6319
Call after 1 pm during working day. Or try reach the boss Mr Kelvin Tok at Pg 9505 1717 (long time no call, not sure still work).

Charges: S$20 per month for SGX, prepaid 6 months interval. Last time they have a starter package of $99. This give you all historical data, and 3 months of trial daily downloading. If you are satisfied with the service, then continue with $120 for next six month, if not, then you can have all the data and stop paying (and of course no more downloading of new data after 3 months as they terminate your account at their server end). Not sure if this started package still available, do call and check.

They use a propriety software to dial using modem to their server for downloading daily data every night, typically after 8 pm. If there is any split or stock adjustment, the s/w will automatically adjust all data on you PC. New counter will also automatically added. If you have problem in your PC later such as harddisk crash and lost all data, you can carry your PC to their office and they will pump all data back for you, free of charge. If you are unable to bring the PC for data pumping in, they can mail you a CD with data at a small fee.