Manfred Touron

Wiki

11 pages about "Wiki"

Cheat Sheet

SSH / Mosh

More SSH awesomeness on https://github.com/moul/awesome-ssh

SSH or mosh into a host and attach or create a named tmux session

ssh SERVER -t "tmux attach -t mysession || tmux new-session -s mysession || /bin/sh"

and with mosh

mosh SERVER -- /bin/sh -c "tmux attach -t mysession || tmux new-session -s mysession || /bin/sh"

replaces SERVER with your SSH target hostname.

Unsafe SSH or Mosh connection (temporary servers: CI)

ssh -o ControlMaster=no -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no SERVER

with Mosh:

mosh -ssh="ssh -o ControlMaster=no -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no" -- SERVER

Non-interactive SSH with password

expect -c "set timeout -1; spawn ssh SERVER; match_max 100000; expect *password:*; send -- PASSWORD\r; interact;"

Networking

Measure network performances and stability with iperf

Install:

  • Install on mac with brew install iperf
  • Install on Linux with sudo apt install iperf

TCP test:

  • Run on the server: iperf -s -p 6666
  • Run on the client: iperf -c ${SERVER} -p 6666

UDP test:

  • Run on the server: iperf -s -p 6666 -u
  • Run on the client: iperf -c ${SERVER} -p 6666 -u -b 900m

traceroute & ping with mytraceroute (mtr)

  • Install on Linux: sudo apt install mtr
  • Usage: mtr ${IP}

Example:

$ mtr google.com
                                   My traceroute  [v0.86]
Keys:  Help   Display mode   Restart statistics   Order of fields   quit
                                                   Packets               Pings
 Host                                            Loss%   Snt   Last   Avg  Best  Wrst StDev
 1. xxx-xxx-xxx-xxx.rev.poneytelecom.eu           0.0%     5    0.4   0.6   0.4   0.7   0.0
 2. 195.154.2.6                                   0.0%     4    0.9   0.9   0.8   1.0   0.0
 3. 209.85.149.12                                 0.0%     4    1.0   1.0   1.0   1.0   0.0
 4. 108.170.245.1                                 0.0%     4    2.4   2.2   2.0   2.4   0.0
 5. 66.249.95.247                                 0.0%     4    1.1   1.1   1.1   1.1   0.0
 6. par21s12-in-f14.1e100.net                     0.0%     4    1.1   1.1   1.1   1.2   0.0

Find big files & Cleanup space

Get aggregated sizes of sub-dirs and files

du -hs .??* *

Find big files

find / -xdev -type f -size +100M -ls 2>/dev/null

Cleanup docker

docker system prune

Git / GitHub

Git / GitHub essential tools

Do rebase instead of merge when pulling

git config --global pull.rebase true

Update the following variables when appropriate: * USER a GitHub user or organization, i.e.: moul

Close a pull request from command line with “hub”

  1. Install “hub”
  2. Type hub api -X PATCH /repos/$USER/$REPO/pulls/$ID -F state=closed

List projects in JSON

  • Fetch: hub api --paginate user/repos | jq . > github.json
  • Only “moul’s” projects: cat github.json | jq '.[] | select(.owner.login == "moul") > moul.json
  • Remove some fields to have a smaller .json file: cat moul.json | jq 'with_entries(select(.key|test("_url")|not)) | del(.owner) | del(.permissions) | del(.url) | del(.node_id)'

Or inline

hub api --paginate user/repos | jq '.[] | select(.owner.login == "moul") | with_entries(select(.key|test("_url")|not)) | del(.owner) | del(.permissions) | del(.url) | del(.node_id)'`

Bonus: find and close a pull-request named “Enabled GuardRails” in the current repository

export prid=$(hub pr list | grep "Enable GuardRails" | cut -d '#' -f 2 | awk '{print $1}')
if [ "$prid" != "" ]; then
    project=$(git remote -v | cut -d: -f2 | sed "s/\\.git.*//" | head -1)
    hub api -X PATCH /repos/$project/pulls/$prid -F state=closed
fi

Manipulate JSON

Flatten concatenated JSON files

Sometimes, when you run a command like this: hub api --paginate user/repos, you get multiple JSON outputs in the same file.

jq doesn’t care and you can run any jq command over the aggregated file, however, this .json file won’t be supported by most of the other softwares.

You can “repair” by flattening a concatenated JSON file by doing the following:

cat aggregate.json | sed 's/^]$$/],/;$$s/],/]]/;1s/^/[/' | jq '. | add'

The sed ... command will transfer the standalones arrays into a a valid array of arrays.

The jq . | add will flatten the array of arrays into a big 1 dimension array.

Apply change on multiple files

Replace a string by another in a whole Git repository

I’m using my https://github.com/moul/golang-repo-template project as repo template on GitHub.

The first task I need to do immediately after creating the new repo, is to replace every instance of golang-repo-template in the codebase by lorem-ipsum (the project name).

for file in $(git grep golang-repo-template | cut -d: -f1 | sort -u); do sed -i'' s/golang-repo-template/lorem-ipsum/g $file; done

Terminator on Windows

  • Install VcXsrv.exe on Windows and configure it to start automatically with the system
  • Create a startTerminator.sh file with the following content: cd; export DISPLAY=:0; nohup terminator &; sleep 1 and change the permissions chmod +x startTerminator.sh
  • Create a shortcut C:\Windows\System32\bash.exe {12345678-1234-5678-0123-456789abcdef} /home/moul/startTerminator.sh

Web

Temporary static web server

python -m SimpleHTTPServer

And then open your browser on http://localhost:8000

Generate directory listing

tree -H . --charset=utf-8 > index.html

Proxy to avoid caching

{ nc ${proxy_addr} ${proxy_port} && exec 1>&- ; } < <(
    awk '{ sub(/GET/, "POST"); print; fflush("") }; NR<2{print "Expires: Thu, 01 Dec 2042 16:00:00 GMT"; print "Cache-Control: public"; }
)

Socat

Connect to a TTY over TCP

socat stdio,raw,echo=0,escape=0x11 tcp-connect:${HOST}:${PORT}

TCP to UDP proxy

socat TCP-LISTEN:9887,reuseaddr,fork UDP:127.0.0.1:9887

This TCP proxy can be forwarded over SSH to have access to an UDP port from localhost from a remote host

Media manipulation

Loop a video with ffmpeg

  1. calculate the amount of loops needed using https://dan.hersam.com/tools/time-calc.html
  2. for i in {1..383}; do printf "file '%s'\n" "loop1.mp4" >> list.txt; done
  3. ffmpeg -f concat -i list.txt -c copy output.mp4

See this status

Contact 🌸

Loading...

P.S. If you’re curious, here’s a “Manual on operating Manfred”, that explains how I operate - following it will make our interactions even more awesome.

Gear

Softwares, Tools, and Services

  • Offline-first tools from this list
  • Code Hosting & Project Management
    • GitHub
    • GitLab, Jira when I’m forced :(
    • ~Waffle (R.I.P)~
    • My own tools: especially depviz
    • AirTable (synced thanks to depviz)
    • Netlify: for hosting (I try to limit all my hosting to static + lambda functions)
    • Docker (and Docker Hub): to manage all my projects (during dev, testing and prodution)
  • Internet Browsing
    • Firefox -> for convenient and privacy browsing
    • Brave -> for experimental, privacy and distributed web
    • Chrome -> for convenient and plugin-friendly browsing
    • Opera -> for temporary sessions
  • Communication
    • Email to maximize the efficiency of providing an information while not interrupt people
    • Slack for long-term communities
    • SMS, Signal, Telegram or even phone calls, when I need to contact someone right now
  • Automate and link services
  • Networking
    • VPN
  • Bookmarking and Offline Reading
    • Pocket
    • I store what I want to read later (and without Internet connection)
    • I setup a zapier script that fills an Airtable doc + I wrote a script that reads that Airtable doc to create bookmark articles on my website
    • Kindle
  • Todolist management
    • Trello, linked with GitHub repos thanks to Unito
  • Development tools
    • Emacs
    • Docker Desktop
    • Postman
  • Banking Accounts:
    • Bunq: Neo-Bank that supports shared accounts
    • Amex: linked with my Bunq Account, mainly used when I need to advance cash for a friend/company so I have the necessary delay to get reimbursed before the money leave my account
    • Paypal: for online payments
  • Cryptography
    • Encfs linked with per-file synchronization services (Google Drive, Dropbox, Git, …)
  • Fonts & Color Profiles

OS X

  • Trello
  • Docker for Mac
  • Encfs
  • Google Drive File Stream
  • Brave
  • Shimo VPN
  • Homebrew
  • Similar to Linux

iOS

  • 1Password
  • Airtable
  • Anaxi
  • Berty
  • Blink SSH
  • Brave
  • Chrome
  • Citymapper
  • Firefox
  • GitHawk
  • Gmail
  • Google Agenda
  • Google Drive
  • Google Maps
  • Google Photos
  • Onion Browser
  • Opera Touch
  • Photos
  • Pocket
  • PomoDoneApp
  • Spotify
  • Slack
  • Trello
  • Working Copy
  • Zenly

Secondary devices (in folders):

  • Banking Accounts
  • Chat Accounts
  • MAO Apps
  • Shazam

Windows

See /windows-10-setup

Linux

  • docker
  • htop
  • encfs
  • assh
  • tmux

Hardwares

  • Computers
    • Macbook Laptops: my most comfortable working environment
    • Windows Laptops: for my windows/linux dev environments, especially useful when doing Open-Source projects and developing appliances
  • Phones and tablets
    • Only iOS mobile/tablet: I don’t trust other systems enough (for now)
    • Microsoft Universal Foldable Keyboard (for iPhone)
    • iPad Smart Keyboard
  • Cryptocurrencies
    • Ledger Nano X + Ledger Nano S as backup

How to audit startup

Pre-audit checklist

  • Hire a freelance to perform general screening (company, staff, external image, etc…)
  • Send set of questions to the founders

Audit checklist

  • Speak with founders to get a macro overview of the project and of the strategy
  • Zoom in interesting details
  • Ask some common questions (see below)
  • Ask founders what kind of advices they would love to have

Common set of questions

  • What will break first with growth?
  • Did you suffer outages? How did that happen?
  • What happen if you have a security breach exploited by a hacker?
  • What happen if one of your employee steal and
  • What are you biggest fears about what could go wrong?
  • Who are your current biggest competitors?
  • Who could become your biggest competitors?
  • What competitive advantage could you take to take advance and discourage any new competitor to enter in the game?
  • You find the genius lamp, what are your 3 wishes?
  • What are your biggest regrets?
  • What are you most proud of?
  • How do you monitor your services?
  • What services/third-parties are you depending on?
  • What is your plan for the next 6 months, 6 years?
  • How developers develop on the product? Do you anonymize dev data? How dev is different from production?
  • What is the typical lifecycle of a feature/an idea?
  • What is the typical lifecycle of a new development?
  • What is the typical lifecycle of a bug fix?

Common topics to review

  • Culture
  • Competitive advantages
  • Most “dangerous” competitors
  • Existing team’s position (will the current helt positions be still appropriate when the organization with scale up?)
  • Team scalability
  • Management
  • Recruitment
  • Privacy
  • Security
  • Monitoring / Alerting
  • Technical Debt / Development comfort
  • Vision
  • Infrastructure scalability
  • Global strategy: aggressiveness, defensive, etc
  • Market size / niche
  • Roadmap quality & realisticness
  • Exit strategy
  • Money raising goal
  • Existing processes and workflows
  • Documentation
  • Technical image/“Sexyness”
  • Feature Development Lifecycle
  • Collaboration between tech/non-tech
  • Management
  • Dev tools / Developer Experience

Post-audit checklist

  • Send report to VCs
  • Debrief if needed
  • Plan to recontact the company to keep updated of the changes

How-to Manfred

Communication

Here are the ways of contacting me, in order of preference.

Please note: I’m open to communication, but I like being effective (though I do enjoy doing some ridiculous and useless things for fun). So feel free to write me, but please get to the point right away :)

  1. Write an email (if you know it) or use the form on my contact page.
  2. Use asynchronous instant messenger (Discord, Matrix, Signal, Slack, Telegram, Twitter, etc).
  3. Use synchronous tools (Voice, Video)

In general:

  • Don’t ask how I’m doing or if I can talk, get to the heart of the matter right away. If it’s inconvenient for me to talk, I’ll just reply later.
  • My communication style is usually dry; don’t take it personally. If you’re cool with keeping the conversation clear and to-the-point, then trust me, I like you a lot :)

Working with partners and collaborators

  • I highly value autonomy and proactivity.
  • You can contact me about any issue and concern, but think of the possible solutions/ideas/interpretations first, and lay them out in your message so that we can discuss it.
  • Trying an idea and failing is ok I will help you move further without any grudges. Being idle and passive is not ok.

If you want to have a meeting with me

I try organizing my time as effectively as possible, so I prefer asynchronicity (in this case - when people can move the subject forward without having to occupy the same space or time). I avoid meetings and calls, a strategy also adopted by some of the famous CEOs, and prefer solving things over email - or chat, for urgent matters.

Oftentimes, the solutions are found at the written discussion stage. But if we do eventually figure that a meeting or a call should take place, as many points as possible should be discussed/planned/researched in advance. We need to be clear on the agenda of the meeting, and have relevant facts and information that can be useful.

Here’s a sample of meeting preparation.

Issue: We need to decide which tools and technologies we’ll be using in the next big project. There are some we’re already efficient with, but there are also some newer, promising ones that we have already tried but didn’t build a project around them.

Initial discussion (asynchronous): A document is created by someone on the team who lists the initial pros/cons/nuances of each technology. It is shared with people who will participate in the development, so that they can write comments, voice their opinions, add information.

Live discussion: Once the doc shows the big picture and the details/opinions, people can exchange their thoughts on Slack, to finalize the decision or to have a real-time debate. The decision is written in the summary of the doc. If they still can’t settle among themselves and need an opinion of an outsider expert, then a meeting is scheduled. (In this case, the main points of the current state of the argument are written in the summary).

Meeting or call with an expert. Beforehand, the doc with all the info and the summary is presented to the expert. The meeting should have as small number of participants as possible (others could listen, but each “camp” should select their representative for the active discussion; the rest should refrain from pitching in as much as possible).