Tianci Hu Marrero
2021-05-30
--
A few months ago, I decided to do my portfolio site in NextJS. Unluckily, I did not think I would add a blog section at the time, and rendering blogs of any other format on the site has just been a pain. (I even tried writing blogs in html, but the finger overhead of formatting in html just dampened my desire to write. This cannot be!) To make matters worse, I store all my blogs on Medium, and migrating them involved many steps:
And some might say, why not just convert medium directly to html? This is because I would potentially want to write my blogs natively instead of publishing them on Medium first. So a future workflow would look like this:
Now, the final workflow described in this post will build on top of an existing solution, which exists in NextJS official site’s tutorial page. If you are familiar with NextJS already, you should start at the “Blog Data” section of “Pre-rendering and Data Fetching” chapter. The tutorial will show you how to:
Though the solution offered by nextJS’ official documentation offered me thearchitecture in which to render my blog, it has the lethal flaw of getting converted html files directly onto the page. I found that the converted html file has trouble with recognizing line breaks (I used spaces, backslashes, and nothing worked consistently). Additionally, there does not seem to be a way to pass in style attributes to the rendered html if your project uses TailwindCSS. This means that I need to stagger the process and render the html files in my directory, so I can go in and tweak them before actual publication.
So, this is my posts.js file, which is based on the next.js documentation:
import fs from 'fs';
import path from 'path';
import matter from 'gray-matter';
import { remark } from 'remark';
import html from 'remark-html';
import {unified} from 'unified'
import remarkParse from 'remark-parse'
import remarkHtml from 'remark-html'
import remarkToc from 'remark-toc'
import remarkRehype from 'remark-rehype'
import rehypeDocument from 'rehype-document'
import rehypeFormat from 'rehype-format'
import rehypeStringify from 'rehype-stringify'
const postsDirectory = path.join(process.cwd(), 'posts');
//for rendering /posts url for a list of posts
export function getSortedPostsData() {
const fileNames = fs.readdirSync(postsDirectory);
const allPostsData = fileNames.map((fileName) => {
//peek into /posts directory in my current working directory
const id = fileName.replace(/\\.md$/, '');
const fullPath = path.join(postsDirectory, fileName);
const fileContents = fs.readFileSync(fullPath, 'utf8');
//fetch the yml metadata from within the markdown files
const matterResult = matter(fileContents);
return {
id,
...matterResult.data,
};
});
// Sort posts by date
return allPostsData.sort((a, b) => {
if (a.date < b.date) {
return 1;
} else {
return -1;
}
});
}
//for rendering /posts url for a list of posts
export async function getPostData(id) {
const fullPath = path.join(postsDirectory, \`${id}.md\`);
const fileContents = fs.readFileSync(fullPath, 'utf8');
const matterResult = matter(fileContents);
let contentHtml;
//find postsHTML directory, if such a file exists, read from such file for rendering posts/\[id\]
const postsHtmlDirectory = path.join(process.cwd(), 'postsHtml');
const htmlPath = path.join(postsHtmlDirectory, \`${id}.html\`);
if (fs.existsSync(htmlPath)) {
contentHtml = fs.readFileSync(htmlPath, 'utf8');
}//if not, create html file with remark, write to html dir, and return newly generated html
else{
const processedContent = await unified()
.use(remarkParse)
.use(remarkToc)
.use(remarkRehype)
.use(rehypeDocument, {title: 'Contents'})
.use(rehypeFormat)
.use(rehypeStringify)
.process(matterResult.content);
contentHtml = processedContent.toString();
fs.writeFile(\`./postsHtml/${id}.html\`, contentHtml, function (err) {})
}
return {
id,
contentHtml,
...matterResult.data,
}
}
export function getAllPostIds() {
const fileNames = fs.readdirSync(postsDirectory);
return fileNames.map((fileName) => {
return {
params: {
id: fileName.replace(/\\.md$/, ''),
},
};
});
}
export async function getStaticPaths() {
const paths = getAllPostIds();
return {
paths,
fallback: false,
};
}
By doing the if/else clause in the getPostData function, I can have a “staggered space” (the /postsHtml directory) where I can tweak the styling of the generated html file with Tailwind.
So now workflow looks at this:
Not perfect, but much better than all the other alternative! And to add a bit more detail on how I convert medium posts into md files, I npm-installed the “medium-to-markdown” package by running npm install medium-to-markdown . And I wrote the following script called mediumConverter.js:
const mediumToMarkdown = require('medium-to-markdown');
const fs = require('fs');
if (process.argv.length === 2) {
console.error('Expected at least one argument!');
process.exit(1);
}
// Enter url here
mediumToMarkdown.convertFromUrl(process.argv\[2\])
.then((markdown) =>
fs.writeFile('./posts/output.md', markdown, function (err) {
console.error(err);})
);
And every time I want to sync my medium post on my site, I just run node mediumConverter.js [medium-post-url]
Another thing I can do is write a script that parses the autogenerated, wacky html, adds tailwind styling based on tag name such as and as well as if a line break is missing. But that’s for another day!