gko::experimental::mpi::
NeighborhoodCommunicator#

Implementation of CollectiveCommunicator that uses MPI-3 graph-topology neighborhood collectives (MPI_Neighbor_alltoallv). Each rank registers only the neighbours it actually exchanges data with, and the MPI runtime is free to schedule the communication accordingly.

Prefer NeighborhoodCommunicator at large rank counts where the communication graph is sparse — typical of distributed PDE discretisations, where each rank only talks to its geometric neighbours. On dense graphs or modest process counts, the default DenseCommunicator is simpler and often just as fast.

class NeighborhoodCommunicator #

Inherits from

A CollectiveCommunicator that uses a neighborhood topology.

The neighborhood communicator is defined by a list of neighbors this rank sends data to and a list of neighbors this rank receives data from. No communication with any ranks that is not in one of those lists will take place.

Public Functions

explicit NeighborhoodCommunicator(communicator base)#

Default constructor with empty communication pattern

Parameters:

base – the base communicator

template<typename LocalIndexType, typename GlobalIndexType>
NeighborhoodCommunicator(
communicator base,
const distributed::index_map<LocalIndexType, GlobalIndexType> &imap,
)#

Create a NeighborhoodCommunicator from an index map.

The receiving neighbors are defined by the remote indices and their owning ranks of the index map. The send neighbors are deduced from that through collective communication.

Template Parameters:

  • LocalIndexType – the local index type of the map

  • GlobalIndexType – the global index type of the map

Parameters:

  • base – the base communicator

  • imap – the index map that defines the communication pattern

virtual std::unique_ptr<CollectiveCommunicator> create_with_same_type(
communicator base,
index_map_ptr imap,
) const override#

Creates a new CollectiveCommunicator with the same dynamic type.

Parameters:

  • base – The base communicator

  • imap – The index_map that defines the communication pattern

Returns:

a CollectiveCommunicator with the same dynamic type

virtual std::unique_ptr<CollectiveCommunicator> create_inverse(
) const override#

Creates the inverse NeighborhoodCommunicator by switching sources and destinations.

Returns:

CollectiveCommunicator with the inverse communication pattern

virtual comm_index_type get_recv_size() const override#

Get the number of elements received by this process within this communication pattern.

Returns:

number of received elements.

virtual comm_index_type get_send_size() const override#

Get the number of elements sent by this process within this communication pattern.

Returns:

number of sent elements.

template<typename SendType, typename RecvType>
request i_all_to_all_v(
std::shared_ptr<const Executor> exec,
const SendType *send_buffer,
RecvType *recv_buffer,
) const#

Non-blocking all-to-all communication.

The send_buffer must have allocated at least get_send_size number of elements, and the recv_buffer must have allocated at least get_recv_size number of elements.

Template Parameters:

  • SendType – the type of the elements to send

  • RecvType – the type of the elements to receive

Parameters:

  • exec – the executor for the communication

  • send_buffer – the send buffer

  • recv_buffer – the receive buffer

Returns:

a request handle

Friends

friend bool operator==(
const NeighborhoodCommunicator &a,
const NeighborhoodCommunicator &b,
)#

Compares two communicators for equality locally.

Equality is defined as having identical or congruent communicators and their communication pattern is equal. No communication is done, i.e. there is no reduction over the local equality check results.

Returns:

true if both communicators are equal.

friend bool operator!=(
const NeighborhoodCommunicator &a,
const NeighborhoodCommunicator &b,
)#

Compares two communicators for inequality.

See also

operator==