<meta charset="utf-8">

<style type="text/css">

p,

li {

white-space: pre-wrap;

}

div {

border-top: 0px;

border-bottom: 0px;

padding-top: 0px;

padding-bottom: 0px;

margin-top: 2em;

}

h3 {

margin-top: 6px;

margin-bottom: 12px;

margin-left: 0px;

margin-right: 0px;

text-indent: 0px;

}

body {

font-family: Arial, Helvetica, sans-serif;

font-size: 7.8pt;

font-weight: 400;

font-style: normal;

}

.large {

font-size: large;

font-weight: 600;

}

.blue {

font-size: large;

font-weight: 600;

color: #0000ff;

}

.red {

font-weight: 600;

color: #aa0000;

}

.green {

font-weight: 600;

color: #00ff00;

}

.explain {

margin-top: 12px;

margin-bottom: 12px;

margin-left: 0px;

margin-right: 0px;

text-indent: 10px;

}

ul {

margin-top: 0px;

margin-bottom: 0px;

margin-left: 0px;

margin-right: 0px;

}

li {

font-size: 8pt;

margin-top: 5px;

margin-bottom: 0px;

margin-left: 0px;

margin-right: 0px;

text-indent: 0px;

}

.li2 {

margin-top: 0px;

}

.li3 {

margin-top: 0px;

margin-bottom: 12px;

}

p.indent {

text-indent: 50px;

}

</style>

<h1>TODO</h1>

<h3>Version 0.93-Alpha.001</h3>

</div>

<div>

<h3> A forming plan </h3>

sqlalchemy implementation is disruptive but I plan to complete it. I plan to benefit from multiple db options,

but its not a free benefit.

The sqlite Numeric columns can be problems. In trade_sum, plan to remove columns that are the results of other

columns. But wait till all

the tables are sqlalchemy declarative models. That should include rr, realrr, sldiff, targdiff, mktval, maxloss.

</div>

<div>

Among the benefits will be a more modular approach to statements enabling a sane and maintainable approach to

including broker statements.

</div>

<div>

<h4>Consider these-- functionality from various commercial apps </h4>

Prelim is to keep focus on how structjour is unique and to stick to its vision of an application to help

day-traders review trading strategy. Its the 'now' focus that is the central focus and strength. Also structjour

will

remain an open source desktop application for a specific audience, which will increase when the focus turns to

supporting

broker statements. (Not ready to begin that work. Need a kind of framework for including statements. This will

begin after the

sqlalchemy models are implemented and used in everything)

</div>

<div>A monthly review mechanism. Don't know how this looks yet.</div>

<div>A list of all trades limited by dates -- basically all the information currently exported to the front page of

disciplined trader displayed as a table. Add to that, green and red visuals, a column of green and red bars,

size = pnl. Think about how to include

r data.

</div>

<div>

<h3>The long planned statistics and long term view-- </h3>

<div>Planning this has been a road block for me as I don't have a clear vision on it. But my murky vision of it

includes many types of charts with

controls to vary the time frame and to filter based on strategy, tags, time of day, day of week. Also equity

over days, months and years needs

to be visually presented. This leaves a mind-boggling number of possible interface approaches.

</div>

<div>

The progress could gain momentum if there were a prior interface and visual plan to roughly follow that

includes the types of charts and

the nature of the controls and filters. Like the development of OpenOffice as based on Word and all. This

would give the statistics project some

needed momentum.

</div>

<div>Statistics on numbers, wins, losses, total pnl based on strategy. Include pie chart showing number of used

strategies and their win ratio/average amount</div>

<div>Some multi-day view of trade notes and daily notes

</div>

<div>Profit calendar is a calendar and navigation tool in which green days are green. Clicking on a day opens

the day

on the front page

</div>

</div>

<ol>

<li> <span class="red">Seperate stock API</span> </span>into a subproject using inheratence

* <span class="red">my_finnhub object </span>and tests

* <span class="red">my_alphavantage object </span>and testss

* <span class="red">my_barchart object </span>and testss

* <span class="red">Test Tiingo. </span>Find its limits and add code to the setLimts

</li>

<li> <span class="red">Add Tiingo to stockAPI. </span>It does everything we require in the free tier.</li>

<li> <span class="red">Implement a filter in myfinnhub. </span>for afterhours. The API always returns afterhours. Structjour has a setting it should observe</li>

<li> <span class="red">output save name is seriously broken. </span>Not sure when it began but it is relying on the text in inputEdit. That is not always the name of the file. Need to find plan carefully/ The output name is associated with both excel and zst files</li>

<li> <span class="red">Alpha Vantage- no longer real time </span> Getting yesterdays data. Refer <a href="https://stackoverflow.com/questions/61278195/fail-to-get-real-time-prices-from-alpha-vantage-api-prices-are-one-day-old">Stack Overflow</a>

* Good news- they seem to be giving a month of data. None of theses changes are documented.

* Stack post got av support answer that prices are frefreshed 4:30 ET. That is prior to now and todays quotes have failed for each stock requested.Yesterday's (Friday) have succeeded.

* They say they are providing extended 4AM to 8PM but it does not seem to be true

and 30 days of data

* Update the rules to ... ??? something that responds to current state I guess. Give it a couple days and tests

</li>

</li>

<li> <span class="red">The time based chart </span>in the statistics hub is not showing the most recenct trades..</li>

<li> <span class="red">The time based chart </span>in the statistics hub is not showing the most recenct trades..</li>

<li> <span class="red">Rewrite graphstuff</span> to use Qt embedded approach where all the graph stuff is

imlmented in a seperate FigureCanvas class and all the data stuff remains in the FinPlot class but depends

more on DB access and less on the TradeSum object </li>

<li> <span class="red">Change all database</span> access to sqlalchemy.</li>

<li> <span class="red"> Update the setup build</span> and include sqlalchemy</li>

<li> <span class="red">Test DefineTrades.fixtsid </span> I dont't think its doing what its supposed to do.</li>

<li> <span class="red">Figure out how to </span>deal with chart data with missing candle-price data from a halt or just low volume. The way after hours is handled is related. Currently entries use the number of candles to find the index. Moving averages... no idea... probably use a time index to remain consistent...Different APIs--some have after hours....What else?</li>

<li> <span class="red">When user browses </span>for and inserts a chart, copy it to the appropriate journal directory.</li>

<li> <span class="red">Provide a details view </span>for each trade showing the transactions as found in Daily Summary but just for the one trade.</li>

<li> <span class="red">Implement a caching </span>of chart data. Save csv in out/.cache directory for each day. Unfortunately will need seperate entries for different APIS.</li>

<li> <span class="red">Finnhub chart data </span>sometimes is weird with all green candles. Seems to happen only on the 1 minute charts. Its like the OHLC vals got sorted. Figure out what is going on</li>

<li> <span class="red">The db entries on my system</span> for TargDiff, SlDiff, RR, MaxLoss, and MstkVal are

inconsistent. Some of them are excel formulas and some are numbers. They should all be numbers or empty.

Excel formulas should only exist up to the point they are used. The most recent date is march 3, 2019 so it

probably does not affect any PyPi distributions, but it should auto correct as statments are saved. ... And

it does already but... I think it lost explain and notes.

<code>

select DISTINCT Date, RR, TargDiff, SLDiff, MaxLoss, MstkVal from trade_sum where RR LIKE "=%" OR TargDiff

LIKE "=%" OR SLDiff LIKE "=%" OR MaxLoss LIKE "=%" OR MstkVal LIKE "=%" ORDER By Date

select DISTINCT Date from trade_sum where RR LIKE "=%" OR TargDiff LIKE "=%" OR SLDiff LIKE "=%" OR MaxLoss

LIKE "=%" OR MstkVal LIKE "=%" ORDER By Date.

</code>

</li>

<li><span class="red">Give a set static </span>time option for one of the charts. (Intended for an all-day 15~30 min chandle chart.</li>

<li> <span class="red">When opening Ib Statement </span>, if using the journal directory, write a DAS style positions.csv to the default location. Add the date to the name, a date column and a column to indicate the source. Note there can only be one positions.csv per statement. Even for a yearly statement</li>

<li> <span class="red">Abstract the journal path </span> from the table chart.path. (Exclude arbitrary path. .... have to update the db. This is a data migration not a model migration. Its temporary for a couple pre-releases then gets removed from migration)</li> <li> <span class="red">Add an account filter. </span>Prereq is the explain entry above. Could allow deleting all trades from an account. (Presumably SIM). Could involve only importing from accounts marked in setting. Could involve only viewing trades from a given account.</li>

<li> <span class="red">Maybe </span>in the entries, make buys green and sells red. Same with PL.</li>

<li> <span class="red">Change </span>the font for explain and notes.</li>

<li> <span class="red">Add </span>some font control for explain and notes.-- probably in context menu.</li>

<li> <span class="red">Create </span>an initialization wizard that leads the user through the initial setup. </li>

<li> <span class="red">Moving average </span>'delayed start' for all apis. (partially done?)</li>

<li> <span class="red">Create a migration </span> change the strategy table to include description columnn --- after sa transition</li>

</ol>

<h3>Version 0.92-Alpha.003</h3>

<h4>Checked in June 3, 2020</h4>

<li> <span class="green">SQlite3.Programming Error </span>about creating using an object from a different thread. It

occurs irregularly.

It happens more frequently when loading a new chart

* The only db call in that code is Manage Keys which happens in ApiChooser and in each api module. The errro happens

irregularly in both.

* Reduced the frequency by storing a dict of keys

* The problem also happens in other places

* it is not affecting the operation of structjour except for long annoying errors-- at least not that I have

discovered

* Seems to be related to the sqlite engine with sqlalchemy and its non-standard thread handling.

* Structjour has frequent new sessions. Each new session is a new transaction in sqlachemy. I think I can rely that

when sa creates a new transaction, the database is flushed and current.

* The problem was calling create_engine with every new session instead of just binding the new session

<code>8/11/20</code>

<li> <span class="green">Finnhub is suddenly </span>not returning premarket. Still returns post market. Check the

parameters I am feeding it.. Did not find a change of policy.

* It seems to be the data. Older (last month) still retrieves pre market <code>8/8/20</code> </li>

<li> <span class="green">Exception in graphStuff </span>. KeyError at 376 ax1.plot(df_ohlc.date, maDict[ma], lw=1,

color=maSetDict[0][ma][1], label=f'{ma}MA').

* Happens with ba fh and av

* Narrowed it down to the MA that are figured as EMAs.

* MAs are figured stock.utilities.movingAvreage. *

* Changing to all Simple Moving averages creates a chart. That code has not changed in months but library versions

(Matplotlib) have.

* The simplest fix is probably massaging the data in utilities.movingAverage for EMA's

* <span class=red>Not going to fix. All MAs return the same calc for an ema now</span>

<code>8/8/20</code>

</li>

<li> <span class="green">Change the layout </span>so that the right widgets expand the right amount.<code>4/6/20</code></li>

<li> <span class="green"> The charts should </span>resize, not keeping perspective, with the widget. <code>4/6/20</code></li>

<li> <span class="green">Add a calendar </span>widget popup to the front dateEdit. <code>4/3/20</code></li>

<li> <span class="green">Move the form files </span>in structjour.view to structjour.view.forms. <code>4/3/20</code></li>

<li> <span class="green"> trade_sum.MktVal</span> has mixed values binary or Numeric and a NOT NULL constraint. Do a one-off data migration to clean up the data<code>4/18/20</code></li>

<li> <span class="green ">Fix realRR to </span> display a percentage of R when the pnl is a loss.<code>4/20/20</code></li>

<li> <span class="green">Time or time zone </span> issues in the finnhub REST API. <code>4/30/20</code></li>

<li> <span class="green">Change Daily Summary </span>to include bar charts to show summaries for the day.

<span class="blue">Implemented two charts, will repeat for each account. Good enough for this TODO list <code>5/1/20</code></span></li>

<li> <span class="green"> Add tags combobox </span>to summaryForm backed by db--simlar to how strategies are done but using sqlalchemy declarative models (first in the program) <code>4/23/20</code></li>

<li> <span class="green"> Move dailyNotes </span>from the dailyForm to summaryForm. <code>4/7/20</code></li>

<li> <span class="green"> Re do dailyForm layout </span>to use splitters.<code>4/5/20</code></li>

<li> <span class="green"> Annotate the charts </span>to show the data source. <code>4/30/20</code></li>

<li> <span class="green">Focus on tags is </span> required for the selected tags to show. I think, unfortunatly, it works as expected.

<span class="Blue"> I 'fixed' it by setting the focus whenever populateTags is called. </span> <code>5/7/20</code </li>

<li> <span class="green">Add a daily pnl </span> label to the front page. <code>5/20/20</code></li>

<li> <span class="green">Create a statistics hub </span>widget to replace the disciplined excel thing.

<span class="blue"> 3 charts proof of concept </span> <code>6/2/20</code></li>

<li> <span class="green">Find the bug</span> 1/10/19 AMD Short does not display PnL. AAPL Long 11/15/2018 at 15:10:51 id=841.

<span class="blue">The causing problem was blob types in pnl. The fix changes the pnl if its a blob or the amount is more than 1 cent off. </span> <code>6/2/20</code></li>

</ol>

<h4>Not going to implement (probably)</h4>

<ul>

<li> <span class="red">Database field order has become</span> inconsistent in trade_sum revealing SELECT * ordererrors. Start a tradesum model class to fix it. (fixed the order errors, and will not be an issue with sa) </li>

</ul>

<h3>Version 0.92-Alpha.001</h3>

<h4>Checked in April 2, 2020</h4>

<ol>

<li> <span class="green">Add backup</span>-restore-initialize utility.</li>

<li> <span class="green">Chart date widget bug. </span>In ChartMage, the widgets are attempting to upgrade but its in a non-gui thread. Move it out to chartMagic1, 2, and 3.</li>

<li> <span class="green">API Chooser </span>should choose a working api. When AV trade failed rules, it did not pick up the next choice. (Can't get it to fail)</li>

<li><span class="green">Create a setting </span>for each api with a datetime value to indicate quota has been reached until a given time. Note that for av, just implement the 5 per minute throttle as you really have to work at it to reach 500 with the throttle. They don't consistently reset the 1 minute time limit. Try 2 minutes. The api will roll on to the next one and this whould minimize the interference. Implement for av, bc, fh and ib </li>

<li><span class="green">Add rules to the </span>api chooser to check for quota</li>

<li><span class="green">Put the apiChooser </span>call in a while loop dependent on apiList having a value. Created a method in APIChooser to do it.</li>

<li> <span class="green">Remove a button. </span>Keep only one save/load button. Will involve automatically changing radio button from DAS or IB to DB after reading a file. Should be able to eliminate a runnitDB method. There are two of them in RunController</li>

<li> <span class="green">Automatically load trade </span>data when the date widget or trade widget is changed and trades are in the db. Prereq: Remove a button</li>

<li> <span class="green">Automatically save trade </span>data when the date widget or trade widget is changed and trades are marked changed.</li>

<li> <span class="green">Add real RR</span> that figures the actual Reward/StopLoss. Some problems. It involves updating the database, and the layout form. The necessary changes have tentacles throughout the app.</li>

<li> <span class="green">Prelim. Create DB Migration procedure </span> The db update should occur before running the program. Will place updates in migrations folder with numbered files. New files for more updates will increment the number and have a descriptive name as to the module where the db table creation is. The updates need to be tied to app version numbers. That also needs to be implemented. For such a small change, this has many steps and they affect future development. It will be minimally django like in preparation for the app to be much more django like. Start thinking about how to move all db stuff into django model classes. Finally with all the changes made in the past week, there are bound to be issues. So I should start a fork in order to keep bug fixes current on the pre migrations code.</li>

<li> <span class="green">Allow user to browse </span>a file in any location. Currently it checks for journal dir. This will require a QMessage info dialog to ask user for the date.</li>

<li> <span class="green">Make the out directory </span>automatic.</li>

<li> <span class="green">Make the naming scheme </span>automatic.</li>

<li> <span class="green">Fix the RTG </span>to generate more realistic trades for thorough testing. Add a positions.csv generator to accompany it. Let it save them with a test name.</li>

<li> <span class="green">When opening DAS </span>statement, only use the default 'positions.csv' if retrieving it under the journal directory.</li>

<li> <span class="green">In Trade Summary </span>table, the balance is represented as dollar amounts.(Error does not happen)</li>

<li> <span class="green">Give default </span>logfile location. Set it automatically if not set</li>

<li> <span class="green">Give default</span>Db location. Set them automatically if not set.</li>

<li> <span class="green">Auto generate</span>the created directories, remove the choice for scheme and place manual creation separate menu item</li>

<li> <span class="green">Implement </span>the inspire quotes at least minimally.</li>

<li> <span class="green">Identify </span>and test all members of lLayoutForm that can be tested automatically </li>

<li> <span class="green">List </span>the othere members and make a list of ways to test them manually.</li>

<li> <span class="green">Connect</span> the Stock API for IB port stuff and the keys</li>

<li> <span class="green">Implement </span>Some kind of logic to load both IB real and Paper ports but set the chosen one active.</li>

<li> <span class="green">Move</span> the Key data from pickle to Database</li>

<li> <span class="green">Fix </span>the IB code to use settings for port and id</li>

<li> <span class="green">Place </span> stock keys in the database from the stockapi dlg</li>

<li> <span class="green">Test </span> settings by deleting them a buch of times. Find all errors from file settings

<ol>

<li> <span class="green">Test </span> settings by deleting them a buch of times. Find all errors from unbalanced settings. </li>

<li> <span class="green">Test </span> settings by deleting them a buch of times. Find all errors from stockAPI </li>

<li> <span class="green">Test </span> settings by deleting them a buch of times. Find all errors from strategy browser</li>

</ol>

</li>

<li> <span class="green">Try </span>to find any possibility of after hours data for barchart and alphavantage </li>

<li> <span class="green">When </span>you are sure (not available probablly), provide infor to user-- maybe tool tips in StockAPI dialog</li>

<li> <span class="green">Create </span>a nice default chart look</li>

<li> <span class="green">Show </span>moving averages</li>

<li> <span class="green">User control </span>over which moving averages and line colors</li>

<li> <span class="green">Add MAs </span>and vwap to charts</li>

<li> <span class="green">Get rid </span>of the data ticks on axis 1</li>

<li> <span class="green">Find a way </span>to set ticks at multipals of the interval and at even numbers by 5s and the like</li>

<li> <span class="green">Set </span>the chart ticks dependent on the interval size</li>

<li> <span class="green">Find a way </span>to set the volume like 1.2M or 350K </li>

<li> <span class="green">Fix </span>the chart entries which seem to be broken</li>

<li> <span class="green">Create </span>a chart user control page</li>

<li> <span class="green">Create </span>the chart dialog to give the some amout of control over charts.</li>

<li> <span class="green">Create </span>a utility for the StockAPI dialog that runs it standalone.</li>

<li> <span class="green">Pop up </span>the stockapi dialog if chart is attempted and no stock api is chosen. Closed. Pop up a query instead.</li>

<li> <span class="green">Implement </span>preferred and not from the strategy browser to the strategy dropdown box..</li>

<li> <span class="green">Implement </span>a trades table as done in escel version</li>

<li> <span class="green">Implement </span>a daily summary as done in excel version</li>

<li> <span class="green">Implement </span>a mistake form as </li>

</ol>

<h3>Not going to implement (probably)</h3>

<ul>

<li> <span class="red">Make chart creation </span>an automatic setting. This might be a bad idea.... </li>

<li> <span class="red">Automatically place</span>Disciplined.xlsx in journal directory</li>

<li> <span class="red">The explain db entry </span>is in the wrong db.</li>

</ul>