Edit Readme for clarity and style

README.md

@@ -9,14 +9,14 @@ tags:
 ---
 
 # Model Description
-ASMTransformers is a project to train and use a machine learning model to compare assembly (arm, amd, risc-v, i386) functions to a database of known functions, 
-to aid in the process of reverse engineering.
+The ASMTransformers model is a machine learning model that calculates embeddings of assembly functions to enable comparison and similarity
+operations. Instruction  set architectures (ISAs) ARM, AMD64, RISC-V and i386 assembly are supported.  The aim is to aid in the process of reverse engineering.
 
 # Status
 Not actively maintained after publishing
 
 # Relevant links
-* [Github repo ASMtransformers](https://github.com/NetherlandsForensicInstitute/asmtransformers)
+* [ASMtransformers on GitHub](https://github.com/NetherlandsForensicInstitute/asmtransformers)
 
 # Version
 2026-08-06
@@ -30,20 +30,15 @@ nor has it been tested on other downstream tasks.
 
 # Architecture description
 
-This model has been trained similarly to [Wang et al.](https://arxiv.org/pdf/2205.12713). We did diverge somewhat from Wang et al. their approach, since our BERT model is trained multilingual.
-We also experimented with different training parameters during the Masked Language Modelling (MLM). For example, we set the 
---mlm-prob parameter to 0.4 in accordance to [Wettig et al., 2023](https://aclanthology.org/2023.eacl-main.217.pdf). We have done some experiments with different mlm-probs 
-(namely 0.15, 0.3 and 0.4) and found that 0.4 yielded the best results. 
+The model is a Bert-base architecture with 12 layers, 12 attention heads and a hidden size of 768. It uses a vocabulary of size 6161 derived from a custom assembler architecture-aware tokenizer.
 
-One remarkable aspect of [Wang et al.](https://arxiv.org/pdf/2205.12713)'s paper, was that they came up with the concept of "Jump target prediction". As part of the 
-MLM procedure, the model also has to predict jump targets: whenever there is a jump in the code, the model shouldn't only predict it to be a 
-jump instruction, but it should also predict to which line of code it's supposed to jump. The idea is that this leads to better understanding of the long-term relations
-in the code, and as Wang et al yielded good results, we have adopted it. 
+The model is trained in two phases - a masked language modeling (MLM) pretraining phase and a fine-tuning phase for function similarity.
 
-Finetuning happens based on triplet loss: given three functions, of which two are similar (see also [Data](https://huggingface.co/NetherlandsForensicInstitute/Multilingual-ASMBERT/blob/main/README.md#data)) and one is deviant, the model has to pick which two functions 
-are similar to each other. We used [Batched Semi Hard Triplet Loss](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#batchsemihardtripletloss) to 
-ensure that the training task is not too easy. 
+The MLM pre-training phase uses a loss function that includes jump target prediction (see [Wang et al.](https://arxiv.org/pdf/2205.12713)). 
+This pretraining was perfomed using a masking probability of 0.4 that was determined through ablation experiments. This is in line with the findings of [Wettig et al., 2023](https://aclanthology.org/2023.eacl-main.217.pdf).
 
+We perform the finetuning phase using [Batched Semi Hard Triplet Loss](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#batchsemihardtripletloss) on batches of size 1024. 
+The training set for this phase includes triplets that are formed using versions of the same function compiled for different instruction set architectures. 
 
 **Estimated training time:**
 
@@ -70,8 +65,6 @@ finetuning: 4 hours on 1 NVIDIA H200
   ]
 ```
 
-**Pretraining parameters used:**
-
 ```
   {
     "architectures": [
@@ -99,7 +92,7 @@ finetuning: 4 hours on 1 NVIDIA H200
   }
 ```
 
-**Preraining parameters used:**
+**Pre-training parameters used:**
 
 ```
   {
@@ -113,21 +106,20 @@ finetuning: 4 hours on 1 NVIDIA H200
   }
 ```
 
-
 # Output
-The model outputs embeddings of size 768 that can be compared using cosine similarity
+The model outputs embeddings of size 768.
 
 # Data
-The dataset is built on the official [Debian Repository](https://wiki.debian.org/DebianRepository). To obtain multiple families of
-assembly, we used `apt` to cross-build the same source package to multiple architectures. The idea is that this gives us 
-the same functions for all four architectures. For all four architectures, these functions are compiled with different optimisation:
-O0, O1, O2, O3, Os and manually selected set with advanced instructions further referenced here as Oc for Optimised-Custom.
-This results in a maximum of 24 (6 optimisation * 4 architectures) different functions 
-which are semantically similar. (i.e. they represent the same functionality but are written differently)
-In practise, it was much easier to obtain amd64 functions than riscv64 functions. Thus, not all functions have 24 semantically similar functions.
+The dataset is derived from the official [Debian Repository](https://wiki.debian.org/DebianRepository). We used the distribution's build and packaging system `apt` to
+cross-compile source packages for multiple ISAs and optimisation levels. We used the standard optimisation levels supported by the distribution's compiler, O0, O1, O2, O3 and Os.
+We also added an optimization level that includes vectorization and cryptographic instructions that we refer to as Optimized-custom, or Oc.
+The binary files from each source package are disassembled using Ghidra and the resulting control-flow graphs (CFGs) are exported.
 
-The dataset is split into a train, test and an evaluation set. This in done on source package, so all binaries and functions belonging to one source package are part of
-either the train or the test set, not both.
+This results in a maximum of 24 (6 optimisation levels and 4 ISAs) potentially CFGs which are semantically identical, i.e. they represent the same functionality.
+However, because not all source packages can be built for all ISAs and because some optimisation levels result in identical code, the actual number is often lower than that.
+
+The dataset is split into a train, test and an evaluation set. This in done on source package basis, so all binaries and functions belonging to one source package are part of
+either the training set or the test set, not both.
 
 **Total amount of functions per architecture**
 
@@ -170,27 +162,15 @@ either the train or the test set, not both.
 ----------------------------
 
 # Preprocessing
-Several preprocessing steps have been taken: CFGs are processed to become a flat token list suitable for a tokenizer or 
-vocabulary builder. This procedure differs slightly between architectures, the processes can be inspected
-[here](https://github.com/NetherlandsForensicInstitute/asmtransformers/tree/main/asmtransformers/asmtransformers/preprocessors).
-
-Some operands have been normalized, 
-to reduce token explosion caused by raw numeric values. These numerical values are expressed as powers of two, such that
-4096 would be 2 ** 12, so `#0x2^c`, so would 6000, but 9000 would be `#0x2^d`. As a consequence, small numbers are distinguishable
-in the vocabulary list, but bigger numbers get one batched representation.
-
-Finally, jump tokens are normalised to be relative `JUMP_ADDR_*` tokens. (* being the line number of the jump adress in the given 
-function, rather than in the full CFG)
-
-
+Several preprocessing steps have been taken. CFGs are processed into a flat token list using ISA aware tokenizers.
+Immediate operands and addresses have been normalized to the nearest power-of-two reduce token explosion caused by raw numeric values.
+Finally, target addresses of jumps, conditional branches or call instructions are normalised to fucntion-relative `JUMP_ADDR_*` tokens. 
 
 # Performance
-Performance was measured in two ways: Mean Reciprocal Rank (MRR) and Accuracy@1. These metrics are useful for this problem 
-since we compare a given embedded function to a database of embedded functions, and then rank the database by cosine similarity.
-Mean Reciprocal rank shows the mean rank (1 the positive example in the database being ranked first, 0.5 when the positive example
-is ranked second, 0.25 when the positive example is ranked fourth, etc). I.e. an MRR of 0.66 shows that the positive example was
-usually ranked first or second. Accuracy@1 shows how many times the positive example ranked first. It is a useful metric, 
-but it's also very strict, which is why we used it combined with MRR.
+We report two performance metrics: Mean Reciprocal Rank (MRR) and Accuracy@1. 
+
+These metrics are suitable for evaluating tasks where a single example is retrieved from a large collection. 
+Accuracy@1 shows how many times the positive example ranked first. It is a useful metric, but it's also very strict, which is why we used it combined with MRR.
 
 |model|ARM64 mrr|ARM64 acc|AMD64 mrr|AMD64 acc|RISC-V mrr|RISC-V acc|I386 mrr|I386 acc|crosslingual mrr|crosslingual acc|
 |-----|---------|---------|---------|---------|----------|----------|--------|--------|----------------|----------------|