gko::experimental::mpi::
DenseCommunicator#

Default implementation of CollectiveCommunicator. Uses the dense collective MPI_Ialltoallv over the standard MPI communicator (no graph topology required; when Ginkgo is configured against Open MPI older than 4.1 it uses the blocking MPI_Alltoallv instead), which makes it portable but means every rank participates in the exchange even if it has no data to send.

Pick DenseCommunicator when in doubt; it works on every communicator topology and performs well at small-to-moderate process counts. Switch to NeighborhoodCommunicator at large rank counts where each rank only communicates with a few neighbours.

class DenseCommunicator #

Inherits from

A CollectiveCommunicator that uses a dense communication.

The dense communicator uses the MPI_Alltoall function for its communication.

Public Functions

explicit DenseCommunicator(communicator base)#

Default constructor with empty communication pattern

Parameters:

base – the base communicator

template<typename LocalIndexType, typename GlobalIndexType>
DenseCommunicator(
communicator base,
const distributed::index_map<LocalIndexType, GlobalIndexType> &imap,
)#

Create a DenseCommunicator from an index map.

The receiving neighbors are defined by the remote indices and their owning ranks of the index map. The send neighbors are deduced from that through collective communication.

Template Parameters:

  • LocalIndexType – the local index type of the map

  • GlobalIndexType – the global index type of the map

Parameters:

  • base – the base communicator

  • imap – the index map that defines the communication pattern

virtual std::unique_ptr<CollectiveCommunicator> create_with_same_type(
communicator base,
index_map_ptr imap,
) const override#

Creates a new CollectiveCommunicator with the same dynamic type.

Parameters:

  • base – The base communicator

  • imap – The index_map that defines the communication pattern

Returns:

a CollectiveCommunicator with the same dynamic type

virtual std::pair<std::unique_ptr<CollectiveCommunicator>, std::vector<comm_index_type>> resize(
std::shared_ptr<const Executor> exec,
const std::vector<comm_index_type> &send_factors,
) const override#

Create new communicator that sends and receives a different number of elements to/from each rank in its current pattern.

The number of elements to send is deduces from the send_factors. If this communicator is used to send 2 elements to rank 0, and 3 elements to rank 1, then send_factors needs to have size 5, which is equal to get_send_size(). An send_factors = { 4, 2, 1, 3, 2} means that instead of the first element to send to rank 0, four elements are sent, instead of the second element to send to rank 0, two elements are sent, and so on. A visual representation of the transformation is: 

[ 1 1 | 1 1 1 ] -> [ 4 2 | 1 3 2] -> [ 1 1 1 1 1 1 | 1 1 1 1 1 1 ]
The communication pattern will not change. Even if send_factors is zero for all elements of one rank, the rank will stay in the new pattern.

Note

This is a collective call.

Parameters:

  • exec – the executor to use for the new communicator

  • send_factors – the new number of elements to send.

Returns:

The new communicator and the recv_factors implied by the send_factors

virtual std::unique_ptr<CollectiveCommunicator> create_inverse(
) const override#

Creates the inverse DenseCommunicator by switching sources and destinations.

Returns:

CollectiveCommunicator with the inverse communication pattern

virtual comm_index_type get_recv_size() const override#

Get the number of elements received by this process within this communication pattern.

Returns:

number of received elements.

virtual comm_index_type get_send_size() const override#

Get the number of elements sent by this process within this communication pattern.

Returns:

number of sent elements.

template<typename SendType, typename RecvType>
request i_all_to_all_v(
std::shared_ptr<const Executor> exec,
const SendType *send_buffer,
RecvType *recv_buffer,
) const#

Non-blocking all-to-all communication.

The send_buffer must have allocated at least get_send_size number of elements, and the recv_buffer must have allocated at least get_recv_size number of elements.

Template Parameters:

  • SendType – the type of the elements to send

  • RecvType – the type of the elements to receive

Parameters:

  • exec – the executor for the communication

  • send_buffer – the send buffer

  • recv_buffer – the receive buffer

Returns:

a request handle

Friends

friend bool operator==(
const DenseCommunicator &a,
const DenseCommunicator &b,
)#

Compares two communicators for equality.

Equality is defined as having identical or congruent communicators and their communication pattern is equal. No communication is done, i.e. there is no reduction over the local equality check results.

Returns:

true if both communicators are equal.

friend bool operator!=(
const DenseCommunicator &a,
const DenseCommunicator &b,
)#

Compares two communicators for inequality.

See also

operator==