projects | Dr Fares Abawi

research

Wrapyfi

Wrapyfi is a middleware communication wrapper for transmitting data across nodes, without altering the operation pipeline of your Python scripts. Wrapyfi introduces a number of helper functions to make middleware integration possible without the need to learn an entire framework, just to parallelize your processes on multiple machines. Wrapyfi supports YARP, ROS, ROS 2, and ZeroMQ.

GASP

GASP is a multimodal social cue integration model for dynamic (video) saliency prediction. It combines the representations of existing social cue detectors (gaze estimation, gaze following, facial expression recognition, and audiovisual saliency prediction) and attends to the most conspicious region.

Prompt-Driven Visuomotor Grasping

A neural model for grasping basic objects with the NICO robot. The initial and target images are fed into the model, along with the shape, color and position of the object to be grasped, and where it should be placed.

OSS

ImageBind LoRA Fine-Tuning

Fine-tuning "ImageBind One Embedding Space to Bind Them All" using Low Rank Adaptation (LoRA)

Distributing the Llama LLM with Wrapyfi

Wrapyfi enables distributing LLaMA (inference only) on multiple GPUs/machines, each with less than 16GB VRAM

demos

Codemirror Lang LaTeX

A Lezer-based LaTeX language extension for CodeMirror 6

FilePizza Client API

Client API for the filePizza WebRTC filesharing server

datasets

Augmented Extended Train Robots Dataset

A dataset containing 3D augmented images of basic shapes organized based on the ETR dataset. The dataset contains instructions for moving a block from one position to another. Basic linguistic instructions, RCL commands, and speech (WaveNet TTS) are provided.

contributions

6DRepNet

Refactored repository for packaging source and correctly linking imports.

al-folio

Added support for video blocks in bib entries.

Goal-Conditioned RL baselines

Added multiple robotic platforms (physical- pianobot. simulated- NICO robot), created a modular generalized model interface, and modified simulation environments.