The Tool Desk
Outbyte Driver Updater FREEFix the driver behind crashes, sound loss and screen glitchesFind Drivers →Outbyte PC Repair FREEClear out junk files and repair common Windows errorsFree Scan →Some links on this page are affiliate links: if you buy through them we may earn a commission, at no extra cost to you.
Perplexity has released open-source networking and inference software designed to run large mixture-of-experts (MoE) workloads across NVIDIA ConnectX and Amazon Web Services’ Elastic Fabric Adapter (EFA). The release, centered on a library called TransferEngine, targets a specific source of NVIDIA lock-in: the networking software and hardware used to move data between GPUs. It is not a new trillion-parameter model, a complete managed AI system, or evidence that NVIDIA GPUs are no longer needed.
What Perplexity released
The software is in Perplexity’s pplx-garden repository, which describes itself as an open-source collection of inference technology and identifies an MIT license. Its fabric-lib component includes an RDMA TransferEngine and point-to-point mixture-of-experts dispatch and combine kernels. The repository also includes Rust and Python components, documentation, tests, Docker material, and benchmark-related directories.
Perplexity’s accompanying paper, posted to arXiv on October 31, 2025, describes the design and evaluation: TransferEngine: Efficient and Portable RDMA Communication for LLM Inference. The software release concerns infrastructure, not model weights: it does not release Kimi K2 or another trillion-parameter model.
Free tools Windows power users keep installed
One-click scans. No signup required.
TransferEngine provides a common interface for two-sided send and receive, one-sided RDMA writes, paged writes for bulk transfers such as KV-cache movement, and immediate-value completion notifications through an ImmCounter mechanism. It can also handle multiple network interface cards (NICs) per GPU. The goal is to let applications use different network hardware without writing a separate communication path for each vendor.
#1 Best Overall
- SWITCH PORTS: 5 -Port 10/100/1000
- SIMPLE: Plug-and-play without a need for IT know-how or support.
- FLEXIBLE: Extensive portfolio provides ultimate flexibility from 5 to 24 ports and PoE combinations
- PERFORMANCE: Gigabit Ethernet and integrated quality-of-service (QoS) intelligence optimize delay-sensitive services and improve overall network performance.
- INNOVATIVE DESIGN: Elegant and compact design, ideal for installation outside of wiring closet such as retail stores, open plan offices, and classrooms
Why MoE workloads need a different communication path
A mixture-of-experts model contains multiple expert submodels. For each input, a routing step sends tokens to selected experts; their results then return to the computation that needs them. When experts sit on different GPUs or nodes, that creates frequent, irregular point-to-point transfers. The performance limit is therefore not just how quickly GPUs perform arithmetic. Data movement across nodes can constrain inference and post-training.
Other large-model workloads also need substantial transfers. Disaggregated inference separates prefill—the processing of an input prompt—from decode, which generates output tokens; the KV cache created during prefill must move to the devices doing decode. Reinforcement-learning and post-training systems may also need to distribute updated model weights. TransferEngine is intended to support these patterns alongside MoE dispatch and combine.
How TransferEngine makes EFA an option
A major difference among networking systems is how they handle the order and completion of transfers. The paper describes NVIDIA ConnectX’s traditional reliable-connection path and AWS EFA’s Scalable Reliable Datagram (SRD) transport as having different ordering behavior. EFA transfers are reliable but are not inherently delivered in order. Software that assumes one fixed ordering model can therefore be difficult to port.
Rank #2
- 【Port Configuration】 Equipped with 24 10/100/1000 Mbps auto-negotiating RJ45 ports with Auto-MDI/MDIX. Easily add more Gigabit Ethernet ports to your mesh systems, routers, and servers
- 【Faster Transmission】 The 24-port switch provides a 4 MB buffer, 48 Gbps switching capacity, and supports 12K jumbo frames to speed up large file transfers and improve overall efficiency
- 【Reliable Data Transfer】 Features IEEE 802.3x flow control to prevent packet loss, ensuring highly reliable and stable data transmission even under heavy network loads
- 【Innovative & Quiet Design】 Full steel housing with ventilation holes on three sides allows efficient heat dissipation. The fanless design ensures noiseless, dust-free operation, fitting perfectly into any business or office
- 【Excellent Lightning Protection】 Built with a professional lightning protection circuit, all ports and the power supply feature 6 kV protection, effectively safeguarding the switch from thunderstorm damage
TransferEngine treats transfers as reliable without assuming they arrive in order, then tracks completion explicitly. It also uses a host-proxy design: instead of having the GPU initiate certain network operations directly, a CPU thread acts as an intermediary. This provides a more portable path across ConnectX and EFA, at the cost of additional CPU, PCIe, and coordination work. Perplexity reports that proxy overhead becomes more noticeable at 64 ranks, so portability does not guarantee the lowest latency for every workload.
AWS describes EFA as a network interface for high-performance computing and distributed workloads; it is not a GPU or a substitute for one. The paper’s tested AWS configurations use four 100-Gbps EFA NICs on p5 instances or two 200-Gbps EFA NICs on p5en instances to reach 400 Gbps. Those are configurations discussed in the paper, not a promise that every AWS accelerator instance has the same networking setup. See AWS’s EFA overview.
What the benchmarks show—and what they do not
Perplexity evaluated nodes with eight NVIDIA H200 GPUs, NVLink, and dual-socket Intel Sapphire Rapids CPUs. The networking configuration was either one 400-Gbps ConnectX-7 adapter or two 200-Gbps EFA NICs. The figures below are the paper authors’ results for those configurations, not independent testing or a general price-performance comparison.
Rank #3
- DEVICE INTERFACE: 8 x Gigabit RJ-45 ports; LED indicators
- TRENDnet LIFETIME PROTECTION: We stand by the quality of our products. The TEG-S83 8-Port Gigabit Desktop Switch is backed by TRENDnet’s Lifetime Manufacturer Protection.
- NDAA and above TAA COMPLIANT: With our NDAA and TAA compliant network switches, you can plan and install networking solutions that Government customers demand today (U.S. and Canada Only)
- RELIABLE TECH SUPPORT: Our team of advisors, support and tech experts are English speaking, and available for all your needs during normal business hours. We take pride in being there for our customers.
- SWITCHING CAPACITY: This 8-port gigabit switch has a 16Gbps switching capacity with a forwarding rate of 11.9Mpps(64-byte packet size)
| Transfer tested | AWS EFA | ConnectX-7 |
|---|---|---|
| 256-KiB single write | 54 Gbps | 116 Gbps |
| 64-KiB paged write | 364 Gbps | 370 Gbps |
| 32-MiB single write | 336 Gbps | 378 Gbps |
The results show why headline bandwidth alone can mislead. In the paper’s tests, single writes generally needed messages of at least 16 MiB to saturate available bandwidth, while paged writes reached saturation with smaller messages. EFA needed larger messages than ConnectX to reach peak performance. The paper also reports that its system reached peak throughput of 400 Gbps on both platforms, but that peak does not mean every message size or application achieves 400 Gbps.
For MoE communication, the authors characterize their implementation as the first viable EFA implementation in their evaluation. On ConnectX-7, they report performance exceeding DeepEP in the tested configurations. In one decode-related comparison, EFA MoE latency was about 30% higher than ConnectX. The paper also reports 1.3-second weight updates for trillion-parameter models using pipelined point-to-point transfers. These are results under the paper’s tested conditions, not guarantees for other models, serving frameworks, instance families, or cluster sizes.
How this differs from NVIDIA’s networking stack
NVIDIA’s GPUDirect RDMA and GPUDirect Async technologies let network devices exchange data with GPU memory while reducing CPU and host-memory overhead. NVIDIA’s ConnectX adapters and related software can provide a tightly optimized path, including GPU-initiated communication. Perplexity’s paper argues that reliance on features such as IBGDA can tie some high-performance MoE implementations to ConnectX hardware. Background on those NVIDIA technologies is available in the GPUDirect RDMA documentation and NVIDIA’s Magnum IO, NVSHMEM, and GPUDirect Async overview.
Rank #4
- SWITCH PORTS: 16 -Port 10/100/1000
- SIMPLE: Plug-and-play without a need for IT know-how or support.
- FLEXIBLE: Extensive portfolio provides ultimate flexibility from 5 to 24 ports and PoE combinations
- PERFORMANCE: Gigabit Ethernet and integrated quality-of-service (QoS) intelligence optimize delay-sensitive services and improve overall network performance.
- INNOVATIVE DESIGN: Elegant and compact design, ideal for installation outside of wiring closet such as retail stores, open plan offices, and classrooms
| Dimension | AWS EFA with TransferEngine | ConnectX with specialized kernels |
|---|---|---|
| Potential advantage | Supports an AWS-native path for the tested distributed workloads | Access to NVIDIA’s optimized networking path and GPU-initiated communication features |
| Transport consideration | SRD is reliable but does not provide inherent in-order delivery | The paper compares against a ConnectX reliable-connection path with different ordering behavior |
| Compute used in Perplexity’s evaluation | NVIDIA H200 GPUs | NVIDIA H200 GPUs |
| Trade-off highlighted by the paper | Host-proxy overhead and higher latency in some tested cases | Dependence on NVIDIA-specific hardware and software for the specialized path |
This distinction matters: the demonstrated portability is between networking options, not GPU vendors. The evaluation uses NVIDIA H200 GPUs on both sides of the comparison. It does not establish support for AMD, Intel, or custom AI accelerators.
What the release does not replace
- NVIDIA GPUs: The paper’s evaluated systems use H200 GPUs; it does not show that they can be removed from the architecture.
- Collective communication libraries: TransferEngine targets irregular point-to-point transfers such as expert routing and KV-cache movement. Collectives remain useful for structured tensor-parallel and data-parallel work; the paper presents these approaches as complementary, not as a wholesale replacement for NCCL.
- A model-serving platform: A public code repository is not a turnkey service for arbitrary models. The available sources do not establish broad production adoption outside Perplexity.
- Infrastructure cost or operational work: The results do not prove that AWS is cheaper than ConnectX-based infrastructure, or that operating a large multi-node system becomes simple. Provisioning, integration, monitoring, and fault handling still matter.
How it compares with other communication projects
Perplexity’s paper places TransferEngine among tools that solve overlapping but not identical problems. Its comparisons are specific to the paper’s 2025 evaluation; software capabilities can change over time.
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
- DeepEP: A MoE communication implementation that the paper describes as strong on ConnectX but dependent on ConnectX-specific IBGDA and
mlx5functionality. Perplexity reports outperforming it in tested ConnectX-7 configurations. - NVSHMEM: Offers flexible NVIDIA-oriented communication primitives. The paper reports substantial performance degradation on EFA for its tested workload. NVIDIA’s technology overview explains its relationship to GPUDirect Async: NVIDIA Magnum IO, NVSHMEM, and GPUDirect Async.
- NIXL: NVIDIA’s inference transfer library. The paper describes EFA support as preliminary in the 2025 version it discusses. See the NIXL repository.
- Mooncake: A project relevant to KV-cache movement and disaggregated inference. The paper says its RDMA TransferEngine did not support EFA at the time of comparison; its conference presentation is available from USENIX FAST ’25.
- UCCL-EP and MSCCL++: Projects relevant to expert-parallel or collective communication optimization. The paper treats them as focused on different or complementary communication patterns rather than direct substitutes for every point-to-point deployment.
Who should evaluate it
TransferEngine is most relevant to teams already considering multi-node MoE inference, disaggregated prefill and decode, large KV-cache transfers, or frequent weight distribution during reinforcement learning and post-training. It may be less useful for small or single-node inference, or a workload dominated by conventional collectives rather than irregular transfers.
Before adopting it, an infrastructure team should verify the fit against its actual deployment:
- Confirm supported GPU instance families, CUDA and driver versions, and the EFA software and libfabric versions required by the chosen environment.
- Check GPU-to-NIC topology, PCIe and NUMA placement, NVLink bandwidth, NIC count, inter-node oversubscription, and the message sizes the workload actually generates.
- Establish whether the target model’s routing implementation matches the published kernels and how the code integrates with the team’s chosen serving stack, such as vLLM, SGLang, TensorRT-LLM, or custom software.
- Benchmark the intended cluster size and measure p50 and p99 dispatch latency, prefill and decode latency separately, tokens per second, GPU and CPU utilization, network utilization, and tail behavior under congestion.
- Test recovery and observability for node or NIC failures, and account for cluster provisioning, checkpointing, and operating expertise—not only transfer throughput.
The cited repository and paper do not provide a complete, version-pinned deployment guide for every stack, so teams should not assume a one-command installation or universal compatibility. Perplexity’s reported limits also include decode-oriented kernel memory overhead and lack of chunking, which restrict some prefill deployments, as well as CPU proxy and dispatch overhead at 64 ranks.
Does this threaten NVIDIA’s dominance?
It challenges a narrower advantage than the headline phrase “NVIDIA’s dominance” suggests. Perplexity has published a credible attempt to make AWS EFA usable for communication patterns that have often favored NVIDIA ConnectX and its software ecosystem. That could give AWS-native teams another option and reduce networking lock-in for some distributed AI workloads.
But the results still rely on NVIDIA GPUs, and the paper does not show that EFA matches ConnectX across all workloads or that customers have broadly moved production systems. The evidence supports a meaningful alternative at the interconnect and communication-software layer—not the end of NVIDIA’s broader position in accelerators, software, or data-center systems.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
