Investigate a bug with git client credentials triggered when you have two sites using git on one account, and delete one of the sites
Fix bug which occurs when you change your username while using the Git client (caused by the broken repo url) and tell Koen and Andre and Josh and Ryan and David
Work out how to pipe custom error messages down
Pipe custom message when you change a blog's handle (breaking repo url)
Fix bug with rename catcher which allows two posts to share a guid. It caused a bug with Disqus comments. To reproduce this bug: Rename an entry. Create new entry with old name. Both entries have same GUID.
Fix bug with two entries with the same guid, possibly related to an entry rename event (i.e. copying the guid across). Make sure that when a rename happens, the old entry is permanently deleted and its guid cannot be reused. I imagine that WRITE foo.txt MV foo.txt bar.txt WRITE bar.txt could trigger this bug.
Fix bug with .emf files in Word Document and follow up with Todd. Can we convert EMFs? Can we pass a flag to pandoc to let them deal with it? Is this just caused by Tiff files? I need a Windows machine to test this.
Fix bug with a once-scheduled post, based on the file's path or previous date metadata. The post remains scheduled even if the date is removed, and even if it is then rebuilt.
Fix bugs with RSS feeds validation. Perhaps extend absoluteURLs helper function? Tell John
Fix bug with date metadata on Vincent's blog and follow up. I suspect it's to do with the ordering of date tokens.
Fix bug with tokens inside blog posts and follow up with Karoly
Fix bug with potentially TeX-related escaping issues and tell Anthony and Roy
Fix issue with variable passed to dateFormat function and tell Leo
Fix bug with image zoomer on mobile devices and tell Alessandro
Fix bug with handling of queries and tell this person.
Fix post encoding bugs
Fix Kevin's URL encoding bug. Each entry's url property is not encoded as a URI component on the server but it needs to be.
Fix encoding bug with greek URLs then tell Rodericus know
Fix bug with template tags in blog post and tell Shibel
Fix bugs with RSS feeds validation. Perhaps extend absoluteURLs helper function? Tell John
do this as a marketing strategy, e.g. target 'blog with microsoft word', 'blog with google docs', 'blog with ia writer' etc... contact the people who make text editors and ask them to link to us?
Personalize the documentation for logged in customers
Add contributor / language and repo information to source code page using GitHub API.
Add a 'Create template' button to dashboard and tell Shawn
Add upload to dashboard
Drag and drop
Add ‘open folder on drive’ link to Client settings page. Same for Dropbox
Redo demo video
Consider switching domain to blot.site
add support for different user subdomain in dev?
set up mail
switch from mailgun to SES over new domain
Integrate font styles rendering into template model:
Remove this behaviour from dashboard/template-editor/save/fonts into update package
Remove font styles from manifesto/manifesto and re-insert them when template is readFromFolder
Add aws cost usage and forecast to daily update email
Resync should produce a report which I get via email
dropbox reset TO BLOT should handle removed folder
dropbox reset FROM BLOT should handle removed folder
dropbox reset TO BLOT should handle revoked token
dropbox reset FROM BLOT should handle revoked token
make google drive reset TO store metadata for case of file/folder name
make google drive reset FROM abortable
make google drive reset TO abortable
make dropbox reset FROM abortable
make dropbox reset TO abortable
Improve Google Drive client:
How do we handle two files with same name and same parent? How does our fileId:path hash handle this? Make sure we delete the right file. Will file.title as opposed to file.name give us the deduped version?
Bug: move file with deduplicated name from sub folder to other folder
download should check for an existing file with same name but different fileId and dedupe before storing the path?
Handle revoked credentials without tanking server
Can the google drive client handle a server restart during setup?
database.folder should throw ENOENT and EEXIST errors for del, set, move etc.
Error handling
Retry when we get google drive's This service is not available error
Handle storage quota errors
Handle 429 errors with exponential backoff retry
Implement folder.rename method to take advantage of Google Drive's true rename feature
Add a way for sync/build to accept a ctime as an option. We can't set this at the operating system level but google drive's API does return a createdTime value and we could use this as the true file's created date, rather than the date the file is made on the server
write should use the best method for large file uploading
Work out if we can test this using Mocks
Improvements to Dropbox client:
Determine if we can hack our way to determining the file upload date on Dropbox using their API?
Parallelize build/download a little more, so posts appear when many files are dropped into folder?
Efficiently handle Dropbox file renames with internal hash of fileIDs
surface errors during setup on dashboard
surface sync issues with Dropbox client on the homepage of dashboard
How do we make errors sticky? To make sure users click through to clients page to re-connect to Dropbox, etc...
run reset-to-blot when sync fails / errors
ensure that when a process dies, any blogs syncing will be reset-to-blot
ensure that when a process dies, any blogs setting up db client will be re-attempted
reconnecting with the same account and same permissions should not create a new folder for full_access=true
make full_folder use the old app folder directory – is there a way to disconnect it from the Blot app? otherwise add a big warning to prevent confusion about where the new folder is when transitioning permissions
handle the download of large files gracefully too
consider adding back in download/upload stream, but add a check to see if the download is stuck then fall back to other method?
add a check for folder storage before doing mass upload
handle folder full error appropriately
handle a sync lock failure on second blog when migrating an app folder
check content hash for upload possible
--
Add support for Indian payments and follow up with Indian customers
Add 'confirm email address' flow to prevent typos in emails
Add simple form endpoint, could be used for contact forms
Add a 'light local server' for template development which zips then posts template files to Blot's endpoint for live-auto updating previews for a given blog – there would need to be authentication with a one-time pass
Add tag to iterate over all images in a folder and tell Laurel
Fix bug with Work out why Pandoc cannot handle 'linked' images (instead of conventionally embedded images). For instructiosn on reproducing bug, see SG's email and follow up when resolved.
Fix bug with tags that contain slashes and tell Jack
Determine why we get zombie processes after running reload server script.
Create starter kits for each template
e.g. a starter kit on the blog template for a writer, or a photographer, or a journalist, or an academic.
these could be downloaded as folders on the template page?
Create an index of customer-created templates and add Matt's theme
Add 'Hidden: yes' metadata to replace 'Menu: no' and tell Sebastien
Add method to merge updates into custom templates and tell Dave
Improve backlink's tolerance
Add support for full qualified URLs in the backlinks list for each entry. For example, if your blog is example.com, the following link will not appear as an internal link: Foo because build/prepare/internalLinks is not aware of the blog domain/handle
cache domain validity and display checkmark on dashboard
validate the domain is pointing to Blot before allowing resty-auto-ssl to request a cert. we seem to get into a DNS mismatch where a client thinks a domain points to Blot, and triggers a cert request through resty-auto-ssl but whatever DNS server Blot's server points to has the stale record (pointing elsewhere)
Disable redirect to HTTPS for blogs by default (to ensure old browsers without SNI work) but ensure all prominent links to the blog on dashboard use HTTPS
Improve clients
cache sync validity and display checkmark on dashboard
surface errors for clients on dashboard index
add a change button next to the dropbox folder selected on the dropbox client where possible
add blocking process to dropbox connection when writing folder contents
ensure the app permission access_token is destroyed when switching to full-folder and vice versa
add a folder chooser during full-folder setup
ensure you can move the folder in full-folder permission mode
Move citations out of {{{html}}} and tell questioner and Mikka
Persist data somehow? import/export from local folder .blot?
Fix 'Visit' link on dashboard
Can we expand this to a simple self-hosting process ('npx blot-server')?
Template engine plan
create data/blogs/blogID/templates directory
show error message on preview-of subdomain
only allow valid templates to get stored in data/blogs/blogID/templates, used for live site?
maybe we only store in redis valid templates, live site pulls
Improve server-side scheduled tasks:
Write task to identify and delete long overdue customers
Fetch dimensions of images even if optimize and cache is disable, so that image zoom can work. Tell Joshua
Add action to lint code with jshint and eslint
any other static code analysis tools?
separate code to check for broken links, run in parallel
Improvements to metadata extractor:
Improve date parser to use time from file creation date if it's within the same day as the date specified in metadata and follow up with Nicolas
Fix summary generator such that a subtitle is used for summary text (i.e. an h2 directly following an h1) We still want to prevent the scenario in which a later h2 becomes the summary, though. Notify Jessica and rebuild Jessica's site
Add support for entry property with escaped double quotes, tell Jay
Improve questions section of site, in approximate order of importance:
Add way to mass-rename tags
Add display aliases for tags e.g. how -> How to use Blot, etc.
Fix breadcrumbs within Questions section
Fix pagination
store version history of the question and answer edits
add option to hide/delete a question
question tagging, including a list of all tags, and popular tags
give the 'ask' and 'answer' textarea inputs the full functionality of the GitHub issue/PR comment inputs (i.e. code snippets / preview / copy/paste screenshots from clipboard)
email notifications (There's an existing system in place for sending emails in app/helper/email)
upvoting for topics and upvote-based ranking on the topic index
show vote count and answer count next to each question on index
add a filter for topic index to show questions by: 'Newest', 'Active', 'Unanswered', 'Votes'
'run code snippet' feature / similar to trinkets.io
Add rel=canonical to all the default templates, or fix the issue that Blot serves the same thing to URLs with a trailing slash, and without.
If we do add rel=canonical, make sure to consider the situation where someone mounts a page to the index of their site, e.g. /page/1 and / will now be different.
Add way to nest tags / hierarchy for tags and tell Uzoamaka
Adjust the chargeForRemaining function in create-blog such that the balance is added or deducted from the proration charge.
Improve transformer so it checks a file's MTIME against a stored value before rehashing the file?
Adjust interface to accept a number of directories. Make it 'unaware' of blog IDs?
Add method which accepts a path or URL and flushes what is stored for it, update the rebuild entry script
Cache errors. When a post contains multiple identical links to images that timeout/404, the thumbnail generator should only make 1 attempt to fetch them. Right now we have a post that is filled with identical broken image links (in this case to a spacer.gif) can suck up the build process for hours.
Add a 'page link format' option to the dashboard – the link format now only applies to entries. Tell Mike
Prevent long-overdue customers from accessing the service, then tell Matvey
Add support for arbritary template folder location – any subfolder could be a template folder if it contains a valid template.json file. Tell Bernardo
Generate proper favicons from uploaded avatars (to prevent streched favicons when non-square avatars are uploaded)
Experiment with GitHub actions setup
Look into potentially useful actions for code quality / linting / automtatic dependency updates
Work out way to recover from expired cert in the cache. To reproduce, consider a member who signed up, connected example.com, then moved the domain to another service. The attempt to renew the cert for example.com will fail. When the customer re-registers with Blot, for some reason open-resty-auto-ssl will serve an old certificate. I followed these steps to reset but we should automate this
NGINX doesn't appear to be following the symlinks created by node.js to serve the contents of each blog's folder and the contents of each blog's static directory. Is this a permissions issue? Is now finally the time to set up users with appropriate permissions for blot, nginx, build etc...
Improve the automatic title generated for screenshots with default MacOs filename from 'at 14:40pm'
Add support for posts generated from Excel files, would generate a table. Use this library?
Use blurhash to generate placeholders when lazy loading images and tell David and Marius
Work out how to cache 404s at the NGINX level
I tried extending express-disk-cache to store images inside /https/blot.im/missing/foo-bar.html but I can't set a 404 status code for the specific tryfiles directive which could match that file.
The application server ends up spending a lot of time serving 404s
Improvements to template engine
Make it possible to change URL of tagged pages and tell Jarrod
Add blog.statistics and entry.statistics (total words, etc.) and tell Pratik
Work out how to make live but saveable modifications for preview purposes on the template editor. I have already established a messaging channel between preview HTML pages and the template editor with the script injected in render/middleware.
Add way to iterate over public and private pages to allow sitemap to work properly and tell Rodrigo and questioner
Fix bug with template editor which resets content of a view. Follow up with Nathan who has extensive steps to reproduce, and John
Allow the name/slug of a template to change and tell Marcus and Pratik and Brad when the name of a locally-edited template in your blog's folder updates
Allow ordering of posts by file name and tell Jacopo
Re-send receipt when the customer changes their email after initial payment, pre-account creation.
Once stripe allows programmatic receipt delivery
Improvements to entry builder
Adjust title metadata extractor to fallback to file name rather than first sentence of file if the post does not contain a title tag. The current behaviour has suprised a number of people. If this breaks John's template create new first_sentence property of entry to use on the template.
Generate a list of titles which you could use to generate a table of contents and tell Zach
Use time draft was moved into blog folder for entry created/publish date, rather than the time the draft file itself was created. Tell Kerim and Shibel
Add way to filter entries by post type (e.g. photo) and tell Justin.
If entries share publish date, sort them by blot creation date
Fix bug with blockquotes and tell Kerim. Multiple ">" symbols on the blank paragraphs between other quoted text are not formatting properly.
Fix bug with indented code in pre tags and tell Michael
Improvements to dashboard
Fix bug with file list on dashboard in Safari and tell Ryan
Fix missing error messages on overdue subscription payment form
Use <select> optgroup on timezone select dropdown to subdivide zones by current time, and steal the presentation format used by basecamp.
Add a broken link checker
Make dashboard requests idempotent to prevent multiple form resubmits, multiple new blogs created accidentally? More information about idempotent requests
Indicate that some posts' permalink will not change (if metadata) on the dashboard when setting the blog's permalink format. Tell Chet.
Dumb quotes are still smartified, even without typeset enabled, possibly by pandoc?
Undo option for form changes
Add transactional emails notifying customer of changes to their email or password.
When a customer pastes the whole tracking script into the preferences page on the dashboard, strip out everything except the ID which we need... perhaps create a page explaining what
is happening before doing this.
Handle noscript on dashboard / signup gracefully (remove all dashboard js?)
Allow customers who are restarting an account with multiple blogs to disable some before restarting.
Extend the Markdown converted to add support for Ulysses highlighting and iAWriter highlighting syntax, using the example files provided by Marco. Tell Marco
Removing Date metadata from a post should reset its date to the file's creation date. Currently nothing changes.
Fall back to file name before first sentence when generating title metadata and tell Rob
Add support for setting the index of an item in the menu to metadata and tell Silvio
Implement hidden link section on Menu page under profile settings. Default links not on the menu would appear there, making it easy to restore them in future. Tell Josh
Add new variable for last-build-time for a blog, and use this for in the sites' RSS feeds. Tell Shawn
Redirect old draft URL to live published post and tell Shibel
Remove git folder when switching to Dropbox client and tell Jamie
Backdate existing commits (perhaps set file ctime using info from git log?) This would sidestep the frustration experienced by Gavin when he imported an existing repository.
Fix bugs with git removed / renamed file unit test on Travis
Extend sync lock during long syncs
Write test to simulate process dying and restarting mid-pull. Does the client recover? What happens if a pull fails midway? Does git recover?
Write tests for a series of random client operations (pull, add, push, move, reset, delete) that can be seeded and reliably re-run in the case of failure.
Move to neat promise chain api when 2.0 comes out
Test that client must keep sending new entries and deleted entries when it recieved an error from entry.build otherwise you get ghost posts...
send nice error message with instructions for 413 request too large: "git reset HEAD^" etc...
Thoroughly document:
Tidy up tests and reduce size and runtime
This client should acquire a lock on the blog's folder before doing stuff to it more generally, especially during initialization etc...
what happens if customer changes their handle? will pushing break?
I want them to be able to keep using their old one until someone else claims it, then I want 401 unauthorized, please reauthenticate with new username... but the remote path will be wrong?
Rate limit authentication route?
Dropbox client
Handle file renames using Dropbox IDs to avoid the 'new entry' bug when you rename a file
Build posts incrementally while big syncs are taking place
Revert lowercase for all files when disconnecting the client
add support for Dropbox Paper files using paperDocsDownload and tell Emad
Add integration test with a dropbox Webhook call. I want to not push some code which would prevent syncing customer folders.
Does client.remove support the removal of folders? If not, make it so and add a test
Add a step to authentication flow which says: 'there is already a folder called 'Your blog title' in your Dropbox, would you like to use that?' then overwrite the files inside. This prevents the proliferation of confusing 'Your blog title (1)' when you re-authenticate.
Add a progress bar to the dashboard when writing existing files to Dropbox. It takes a while and confuses people.
Adding an additional blog to existing app folder under constant webhooks failed. New blog had correct folder but old blog still used root. I guess we need to acquire a lock on ALL blogs interacting with that dropbox account.
Extend sync lock during long syncs
Write test to simulate process dying and restarting mid-pull. Does the client recover?
Write tests for a series of random dropbox client operations (pull, add, push, move, reset, delete) that can be seeded and reliably re-run in the case of failure.
Can we write to a folder_id instead of a path? It would be nice to be able to move the blog's folder even when writing the initial files to it, for example...
This should be simple once we copy the exisiting state of the blog folder across. But right now it doesn't really do much. It resets the local server folder. Then resyncs.
Update to latest version of Dropbox client.
Make the folder in Dropbox rename automatically when the
customer changes the username for one of their multiple blogs?
Add media-query to prevent typeset hanging punctuation on small screens (it looks weird) and tell Shibel
Move to using unix socket with redis client per documentation for better throughput. Read this issue first, I might need to make some user group permissions tweaks to ensure things are secure.
Move blog's folder contents from /blogs/{blogID} to data/blogs/{blogID}/folder
Move static files directory from /static/{blogID} to /data/blogs/{blogID}/static
Move db directory from /db to /data/db
Remove /tmp directory and instead use os.tmpDir
Switch to streaming JSON parser. There are npm modules that offer asynchronous JSON APIs. See for example:
JSONStream, which has stream APIs.
Big-Friendly JSON, which has stream APIs as well as asynchronous versions of the standard JSON APIs using the partitioning-on-the-Event-Loop paradigm outlined below.
Add notification emails for monit events
Include secure one-time link to reboot the AWS instance?
If Apple releases an API, tell people who are interested in using the iCloud client
Could we have multiple syncing method attached to one blog?
Generate list of external API dependencies and follow changes for each
Add simple minimal version of Blot that could be npm installed, then tell Pedro who is frustrated editing templates locally with the git client
Subscription refactor
Different payment methods per blog
Seperate blog subscriptions behind the scenes to avoid the crazy calculations. Think of them like domains, you wouldn't do what Blot is doing for those...
Create script to export Blot sites to common formats like Wordpress and let Roberto know
Move to Linode?
Preserve AWS elastic IP and proxy to Linode
Think about eventual multiple locations around the world from day 1
Allow logged in customers to see public homepage. Perhaps use dashboard.blot.im for dashboard?
Add a slack.com style dropdown on the public homepage for logged in customers which takes them to the dashboard for their sites.
When the customer's blog does not have a client, client.write and client.remove need to remove the files from the source folder. Create some kind of local client which we fall back and use when none is configured?
Work out how to host podcasts on Blot and tell Adrian
Client.write does not propogate sync errors... to demonstrate this, revoke dropbox token then try to enable local editing.
Re-enable image minifier and fix the bugs it creates
Add support for fetching only some properties of an entry (ideally only those that are used). prevent some properties from being fetched in some lists (html, for example in all_entries). Would be a good candidate for hgetall or hget?
Embed feature
Add support for native PDF embeds in posts, just put a link to a PDF or a path, e.g. /_assets/file.pdf --> would become a nice embed.
Add support for native MP3 embeds in posts, just put a link to an MP3 or a path, e.g. /_assets/file.mp3 --> would become a nice embed then tell Luke and Nash and Andy and Charlie and Konstantin
Expose exif data on Photo posts? See if you can add comments to image metadata and tell Riley, also tell Mike
Improvements to thumbnail generator
Prioritize faces in crop?
Try to crop out ruined borders. E.g. when making a square crop of a rectangular image with a border, crop into the image as well to remove the border-on-three-sides effect
Extract color palette too for the image
Offer imgix style API for embedding thumbnails
Improvements to reverse proxy:
Ensure I am doing request coalescing on NGINX and lower proxy timeout limit