``cmpxchg`` and ``atomicrmw`` are essentially like an atomic load followed by an
atomic store (where the store is conditional for ``cmpxchg``), but no other
-memory operation can happen on any thread between the load and store. Note that
-LLVM's cmpxchg does not provide quite as many options as the C++0x version.
+memory operation can happen on any thread between the load and store.
A ``fence`` provides Acquire and/or Release ordering which is not part of
another operation; it is normally used along with Monotonic memory operations.
on the users of the result, some ``atomicrmw`` operations can be translated into
operations like ``LOCK AND``, but that does not work in general.
-On ARM, MIPS, and many other RISC architectures, Acquire, Release, and
-SequentiallyConsistent semantics require barrier instructions for every such
+On ARM (before v8), MIPS, and many other RISC architectures, Acquire, Release,
+and SequentiallyConsistent semantics require barrier instructions for every such
operation. Loads and stores generate normal instructions. ``cmpxchg`` and
``atomicrmw`` can be represented using a loop with LL/SC-style instructions
which take some sort of exclusive lock on a cache line (``LDREX`` and ``STREX``
-on ARM, etc.). At the moment, the IR does not provide any way to represent a
-weak ``cmpxchg`` which would not require a loop.
+on ARM, etc.).
::
- cmpxchg [volatile] <ty>* <pointer>, <ty> <cmp>, <ty> <new> [singlethread] <success ordering> <failure ordering> ; yields {ty}
+ cmpxchg [weak] [volatile] <ty>* <pointer>, <ty> <cmp>, <ty> <new> [singlethread] <success ordering> <failure ordering> ; yields { <ty>, i1 }
Overview:
"""""""""
The '``cmpxchg``' instruction is used to atomically modify memory. It
loads a value in memory and compares it to a given value. If they are
-equal, it stores a new value into the memory.
+equal, it tries to store a new value into the memory.
Arguments:
""""""""""
Semantics:
""""""""""
-The contents of memory at the location specified by the '``<pointer>``'
-operand is read and compared to '``<cmp>``'; if the read value is the
-equal, '``<new>``' is written. The original value at the location is
-returned.
+The contents of memory at the location specified by the '``<pointer>``' operand
+is read and compared to '``<cmp>``'; if the read value is the equal, the
+'``<new>``' is written. The original value at the location is returned, together
+with a flag indicating success (true) or failure (false).
+
+If the cmpxchg operation is marked as ``weak`` then a spurious failure is
+permitted: the operation may not write ``<new>`` even if the comparison
+matched.
+
+If the cmpxchg operation is strong (the default), the i1 value is 1 if and only
+if the value loaded equals ``cmp``.
A successful ``cmpxchg`` is a read-modify-write instruction for the purpose of
identifying release sequences. A failed ``cmpxchg`` is equivalent to an atomic
.. code-block:: llvm
entry:
- %orig = atomic load i32* %ptr unordered ; yields {i32}
+ %orig = atomic load i32* %ptr unordered ; yields i32
br label %loop
loop:
%cmp = phi i32 [ %orig, %entry ], [%old, %loop]
%squared = mul i32 %cmp, %cmp
- %old = cmpxchg i32* %ptr, i32 %cmp, i32 %squared acq_rel monotonic ; yields {i32}
- %success = icmp eq i32 %cmp, %old
+ %val_success = cmpxchg i32* %ptr, i32 %cmp, i32 %squared acq_rel monotonic ; yields { i32, i1 }
+ %value_loaded = extractvalue { i32, i1 } %val_success, 0
+ %success = extractvalue { i32, i1 } %val_success, 1
br i1 %success, label %done, label %loop
done:
/// This corresponds to the cmpxchg instruction.
ATOMIC_CMP_SWAP,
+ /// Val, Success, OUTCHAIN
+ /// = ATOMIC_CMP_SWAP_WITH_SUCCESS(INCHAIN, ptr, cmp, swap)
+ /// N.b. this is still a strong cmpxchg operation, so
+ /// Success == "Val == cmp".
+ ATOMIC_CMP_SWAP_WITH_SUCCESS,
+
/// Val, OUTCHAIN = ATOMIC_SWAP(INCHAIN, ptr, amt)
/// Val, OUTCHAIN = ATOMIC_LOAD_[OpName](INCHAIN, ptr, amt)
/// For double-word atomic operations:
SDValue getVAArg(EVT VT, SDLoc dl, SDValue Chain, SDValue Ptr,
SDValue SV, unsigned Align);
- /// getAtomic - Gets a node for an atomic op, produces result and chain and
- /// takes 3 operands
- SDValue getAtomic(unsigned Opcode, SDLoc dl, EVT MemVT, SDValue Chain,
- SDValue Ptr, SDValue Cmp, SDValue Swp,
- MachinePointerInfo PtrInfo, unsigned Alignment,
- AtomicOrdering SuccessOrdering,
- AtomicOrdering FailureOrdering,
- SynchronizationScope SynchScope);
- SDValue getAtomic(unsigned Opcode, SDLoc dl, EVT MemVT, SDValue Chain,
- SDValue Ptr, SDValue Cmp, SDValue Swp,
- MachineMemOperand *MMO,
- AtomicOrdering SuccessOrdering,
- AtomicOrdering FailureOrdering,
- SynchronizationScope SynchScope);
+ /// getAtomicCmpSwap - Gets a node for an atomic cmpxchg op. There are two
+ /// valid Opcodes. ISD::ATOMIC_CMO_SWAP produces a the value loaded and a
+ /// chain result. ISD::ATOMIC_CMP_SWAP_WITH_SUCCESS produces the value loaded,
+ /// a success flag (initially i1), and a chain.
+ SDValue getAtomicCmpSwap(unsigned Opcode, SDLoc dl, EVT MemVT, SDVTList VTs,
+ SDValue Chain, SDValue Ptr, SDValue Cmp, SDValue Swp,
+ MachinePointerInfo PtrInfo, unsigned Alignment,
+ AtomicOrdering SuccessOrdering,
+ AtomicOrdering FailureOrdering,
+ SynchronizationScope SynchScope);
+ SDValue getAtomicCmpSwap(unsigned Opcode, SDLoc dl, EVT MemVT, SDVTList VTs,
+ SDValue Chain, SDValue Ptr, SDValue Cmp, SDValue Swp,
+ MachineMemOperand *MMO,
+ AtomicOrdering SuccessOrdering,
+ AtomicOrdering FailureOrdering,
+ SynchronizationScope SynchScope);
/// getAtomic - Gets a node for an atomic op, produces result (if relevant)
/// and chain and takes 2 operands.
N->getOpcode() == ISD::STORE ||
N->getOpcode() == ISD::PREFETCH ||
N->getOpcode() == ISD::ATOMIC_CMP_SWAP ||
+ N->getOpcode() == ISD::ATOMIC_CMP_SWAP_WITH_SUCCESS ||
N->getOpcode() == ISD::ATOMIC_SWAP ||
N->getOpcode() == ISD::ATOMIC_LOAD_ADD ||
N->getOpcode() == ISD::ATOMIC_LOAD_SUB ||
bool isCompareAndSwap() const {
unsigned Op = getOpcode();
- return Op == ISD::ATOMIC_CMP_SWAP;
+ return Op == ISD::ATOMIC_CMP_SWAP || Op == ISD::ATOMIC_CMP_SWAP_WITH_SUCCESS;
}
// Methods to support isa and dyn_cast
static bool classof(const SDNode *N) {
return N->getOpcode() == ISD::ATOMIC_CMP_SWAP ||
+ N->getOpcode() == ISD::ATOMIC_CMP_SWAP_WITH_SUCCESS ||
N->getOpcode() == ISD::ATOMIC_SWAP ||
N->getOpcode() == ISD::ATOMIC_LOAD_ADD ||
N->getOpcode() == ISD::ATOMIC_LOAD_SUB ||
(unsigned)V);
}
+ /// Return true if this cmpxchg may spuriously fail.
+ bool isWeak() const {
+ return getSubclassDataFromInstruction() & 0x100;
+ }
+
+ void setWeak(bool IsWeak) {
+ setInstructionSubclassData((getSubclassDataFromInstruction() & ~0x100) |
+ (IsWeak << 8));
+ }
+
/// Transparently provide more efficient getOperand methods.
DECLARE_TRANSPARENT_OPERAND_ACCESSORS(Value);
KEYWORD(available_externally);
KEYWORD(linkonce);
KEYWORD(linkonce_odr);
- KEYWORD(weak);
+ KEYWORD(weak); // Use as a linkage, and a modifier for "cmpxchg".
KEYWORD(weak_odr);
KEYWORD(appending);
KEYWORD(dllimport);
}
/// ParseCmpXchg
-/// ::= 'cmpxchg' 'volatile'? TypeAndValue ',' TypeAndValue ',' TypeAndValue
-/// 'singlethread'? AtomicOrdering AtomicOrdering
+/// ::= 'cmpxchg' 'weak'? 'volatile'? TypeAndValue ',' TypeAndValue ','
+/// TypeAndValue 'singlethread'? AtomicOrdering AtomicOrdering
int LLParser::ParseCmpXchg(Instruction *&Inst, PerFunctionState &PFS) {
Value *Ptr, *Cmp, *New; LocTy PtrLoc, CmpLoc, NewLoc;
bool AteExtraComma = false;
AtomicOrdering FailureOrdering = NotAtomic;
SynchronizationScope Scope = CrossThread;
bool isVolatile = false;
+ bool isWeak = false;
+
+ if (EatIfPresent(lltok::kw_weak))
+ isWeak = true;
if (EatIfPresent(lltok::kw_volatile))
isVolatile = true;
return Error(NewLoc, "cmpxchg operand must be power-of-two byte-sized"
" integer");
- AtomicCmpXchgInst *CXI = new AtomicCmpXchgInst(Ptr, Cmp, New, SuccessOrdering,
- FailureOrdering, Scope);
+ AtomicCmpXchgInst *CXI = new AtomicCmpXchgInst(
+ Ptr, Cmp, New, SuccessOrdering, FailureOrdering, Scope);
CXI->setVolatile(isVolatile);
+ CXI->setWeak(isWeak);
Inst = CXI;
return AteExtraComma ? InstExtraComma : InstNormal;
}
kw_linker_private, // NOTE: deprecated, for parser compatibility
kw_linker_private_weak, // NOTE: deprecated, for parser compatibility
kw_linkonce, kw_linkonce_odr,
- kw_weak, kw_weak_odr, kw_appending,
+ kw_weak, // Used as a linkage, and a modifier for "cmpxchg".
+ kw_weak_odr, kw_appending,
kw_dllimport, kw_dllexport, kw_common, kw_available_externally,
kw_default, kw_hidden, kw_protected,
kw_unnamed_addr,
}
case bitc::FUNC_CODE_INST_CMPXCHG: {
// CMPXCHG:[ptrty, ptr, cmp, new, vol, successordering, synchscope,
- // failureordering]
+ // failureordering?, isweak?]
unsigned OpNum = 0;
Value *Ptr, *Cmp, *New;
if (getValueTypePair(Record, OpNum, NextValueNo, Ptr) ||
cast<PointerType>(Ptr->getType())->getElementType(), Cmp) ||
popValue(Record, OpNum, NextValueNo,
cast<PointerType>(Ptr->getType())->getElementType(), New) ||
- (OpNum + 3 != Record.size() && OpNum + 4 != Record.size()))
+ (Record.size() < OpNum + 3 || Record.size() > OpNum + 5))
return Error(InvalidRecord);
AtomicOrdering SuccessOrdering = GetDecodedOrdering(Record[OpNum+1]);
if (SuccessOrdering == NotAtomic || SuccessOrdering == Unordered)
I = new AtomicCmpXchgInst(Ptr, Cmp, New, SuccessOrdering, FailureOrdering,
SynchScope);
cast<AtomicCmpXchgInst>(I)->setVolatile(Record[OpNum]);
+
+ if (Record.size() < 8) {
+ // Before weak cmpxchgs existed, the instruction simply returned the
+ // value loaded from memory, so bitcode files from that era will be
+ // expecting the first component of a modern cmpxchg.
+ CurBB->getInstList().push_back(I);
+ I = ExtractValueInst::Create(I, 0);
+ } else {
+ cast<AtomicCmpXchgInst>(I)->setWeak(Record[OpNum+4]);
+ }
+
InstructionList.push_back(I);
break;
}
cast<AtomicCmpXchgInst>(I).getSynchScope()));
Vals.push_back(GetEncodedOrdering(
cast<AtomicCmpXchgInst>(I).getFailureOrdering()));
+ Vals.push_back(cast<AtomicCmpXchgInst>(I).isWeak());
break;
case Instruction::AtomicRMW:
Code = bitc::FUNC_CODE_INST_ATOMICRMW;
// Setup the builder so we can create any PHIs we need.
Builder.SetInsertPoint(FailureBB, FailureBB->begin());
BasicBlock *SuccessBB = FailureOrder == Monotonic ? BarrierBB : TryStoreBB;
- PHINode *Success = nullptr, *Failure = nullptr;
+ PHINode *Success = Builder.CreatePHI(Type::getInt1Ty(Ctx), 2);
+ Success->addIncoming(ConstantInt::getTrue(Ctx), SuccessBB);
+ Success->addIncoming(ConstantInt::getFalse(Ctx), LoopBB);
// Look for any users of the cmpxchg that are just comparing the loaded value
// against the desired one, and replace them with the CFG-derived version.
+ SmallVector<ExtractValueInst *, 2> PrunedInsts;
for (auto User : CI->users()) {
- ICmpInst *ICmp = dyn_cast<ICmpInst>(User);
- if (!ICmp)
+ ExtractValueInst *EV = dyn_cast<ExtractValueInst>(User);
+ if (!EV)
continue;
- // Because we know ICmp uses CI, we only need one operand to be the old
- // value.
- if (ICmp->getOperand(0) != CI->getCompareOperand() &&
- ICmp->getOperand(1) != CI->getCompareOperand())
- continue;
+ assert(EV->getNumIndices() == 1 && EV->getIndices()[0] <= 1 &&
+ "weird extraction from { iN, i1 }");
- if (ICmp->getPredicate() == CmpInst::ICMP_EQ) {
- if (!Success) {
- Success = Builder.CreatePHI(Type::getInt1Ty(Ctx), 2);
- Success->addIncoming(ConstantInt::getTrue(Ctx), SuccessBB);
- Success->addIncoming(ConstantInt::getFalse(Ctx), LoopBB);
- }
- ICmp->replaceAllUsesWith(Success);
- } else if (ICmp->getPredicate() == CmpInst::ICMP_NE) {
- if (!Failure) {
- Failure = Builder.CreatePHI(Type::getInt1Ty(Ctx), 2);
- Failure->addIncoming(ConstantInt::getFalse(Ctx), SuccessBB);
- Failure->addIncoming(ConstantInt::getTrue(Ctx), LoopBB);
- }
- ICmp->replaceAllUsesWith(Failure);
- }
+ if (EV->getIndices()[0] == 0)
+ EV->replaceAllUsesWith(Loaded);
+ else
+ EV->replaceAllUsesWith(Success);
+
+ PrunedInsts.push_back(EV);
}
- CI->replaceAllUsesWith(Loaded);
- CI->eraseFromParent();
+ // We can remove the instructions now we're no longer iterating through them.
+ for (auto EV : PrunedInsts)
+ EV->eraseFromParent();
+ if (!CI->use_empty()) {
+ // Some use of the full struct return that we don't understand has happened,
+ // so we've got to reconstruct it properly.
+ Value *Res;
+ Res = Builder.CreateInsertValue(UndefValue::get(CI->getType()), Loaded, 0);
+ Res = Builder.CreateInsertValue(Res, Success, 1);
+
+ CI->replaceAllUsesWith(Res);
+ }
+
+ CI->eraseFromParent();
return true;
}
case ISD::ATOMIC_LOAD: {
// There is no libcall for atomic load; fake it with ATOMIC_CMP_SWAP.
SDValue Zero = DAG.getConstant(0, Node->getValueType(0));
- SDValue Swap = DAG.getAtomic(ISD::ATOMIC_CMP_SWAP, dl,
- cast<AtomicSDNode>(Node)->getMemoryVT(),
- Node->getOperand(0),
- Node->getOperand(1), Zero, Zero,
-
cast<AtomicSDNode>(Node)->getMemOperand(),
-
cast<AtomicSDNode>(Node)->getOrdering(),
-
cast<AtomicSDNode>(Node)->getOrdering(),
-
cast<AtomicSDNode>(Node)->getSynchScope());
+ SDVTList VTs = DAG.getVTList(Node->getValueType(0), MVT::Other);
+ SDValue Swap = DAG.getAtomicCmpSwap(
+ ISD::ATOMIC_CMP_SWAP, dl, cast<AtomicSDNode>(Node)->getMemoryVT(), VTs,
+ Node->getOperand(0), Node->getOperand(1), Zero, Zero,
+ cast<AtomicSDNode>(Node)->getMemOperand(),
+ cast<AtomicSDNode>(Node)->getOrdering(),
+ cast<AtomicSDNode>(Node)->getOrdering(),
+ cast<AtomicSDNode>(Node)->getSynchScope());
Results.push_back(Swap.getValue(0));
Results.push_back(Swap.getValue(1));
break;
Results.push_back(Tmp.second);
break;
}
+ case ISD::ATOMIC_CMP_SWAP_WITH_SUCCESS: {
+ // Expanding an ATOMIC_CMP_SWAP_WITH_SUCCESS produces an ATOMIC_CMP_SWAP and
+ // splits out the success value as a comparison. Expanding the resulting
+ // ATOMIC_CMP_SWAP will produce a libcall.
+ SDVTList VTs = DAG.getVTList(Node->getValueType(0), MVT::Other);
+ SDValue Res = DAG.getAtomicCmpSwap(
+ ISD::ATOMIC_CMP_SWAP, dl, cast<AtomicSDNode>(Node)->getMemoryVT(), VTs,
+ Node->getOperand(0), Node->getOperand(1), Node->getOperand(2),
+ Node->getOperand(3), cast<MemSDNode>(Node)->getMemOperand(),
+ cast<AtomicSDNode>(Node)->getSuccessOrdering(),
+ cast<AtomicSDNode>(Node)->getFailureOrdering(),
+ cast<AtomicSDNode>(Node)->getSynchScope());
+
+ SDValue Success = DAG.getSetCC(SDLoc(Node), Node->getValueType(1),
+ Res, Node->getOperand(2), ISD::SETEQ);
+
+ Results.push_back(Res.getValue(0));
+ Results.push_back(Success);
+ Results.push_back(Res.getValue(1));
+ break;
+ }
case ISD::DYNAMIC_STACKALLOC:
ExpandDYNAMIC_STACKALLOC(Node, Results);
break;
Res = PromoteIntRes_Atomic1(cast<AtomicSDNode>(N)); break;
case ISD::ATOMIC_CMP_SWAP:
- Res = PromoteIntRes_Atomic2(cast<AtomicSDNode>(N)); break;
+ case ISD::ATOMIC_CMP_SWAP_WITH_SUCCESS:
+ Res = PromoteIntRes_AtomicCmpSwap(cast<AtomicSDNode>(N), ResNo);
+ break;
}
// If the result is null then the sub-method took care of registering it.
return Res;
}
-SDValue DAGTypeLegalizer::PromoteIntRes_Atomic2(AtomicSDNode *N) {
+SDValue DAGTypeLegalizer::PromoteIntRes_AtomicCmpSwap(AtomicSDNode *N,
+ unsigned ResNo) {
+ if (ResNo == 1) {
+ assert(N->getOpcode() == ISD::ATOMIC_CMP_SWAP_WITH_SUCCESS);
+ EVT SVT = getSetCCResultType(N->getOperand(2).getValueType());
+ EVT NVT = TLI.getTypeToTransformTo(*DAG.getContext(), N->getValueType(1));
+
+ // Only use the result of getSetCCResultType if it is legal,
+ // otherwise just use the promoted result type (NVT).
+ if (!TLI.isTypeLegal(SVT))
+ SVT = NVT;
+
+ SDVTList VTs = DAG.getVTList(N->getValueType(0), SVT, MVT::Other);
+ SDValue Res = DAG.getAtomicCmpSwap(
+ ISD::ATOMIC_CMP_SWAP_WITH_SUCCESS, SDLoc(N), N->getMemoryVT(), VTs,
+ N->getChain(), N->getBasePtr(), N->getOperand(2), N->getOperand(3),
+ N->getMemOperand(), N->getSuccessOrdering(), N->getFailureOrdering(),
+ N->getSynchScope());
+ ReplaceValueWith(SDValue(N, 0), Res.getValue(0));
+ ReplaceValueWith(SDValue(N, 2), Res.getValue(2));
+ return Res.getValue(1);
+ }
+
SDValue Op2 = GetPromotedInteger(N->getOperand(2));
SDValue Op3 = GetPromotedInteger(N->getOperand(3));
- SDValue Res = DAG.getAtomic(N->getOpcode(), SDLoc(N), N->getMemoryVT(),
- N->getChain(), N->getBasePtr(), Op2, Op3,
- N->getMemOperand(), N->getSuccessOrdering(),
- N->getFailureOrdering(), N->getSynchScope());
+ SDVTList VTs =
+ DAG.getVTList(Op2.getValueType(), N->getValueType(1), MVT::Other);
+ SDValue Res = DAG.getAtomicCmpSwap(
+ N->getOpcode(), SDLoc(N), N->getMemoryVT(), VTs, N->getChain(),
+ N->getBasePtr(), Op2, Op3, N->getMemOperand(), N->getSuccessOrdering(),
+ N->getFailureOrdering(), N->getSynchScope());
// Legalized the chain result - switch anything that used the old chain to
// use the new one.
- ReplaceValueWith(SDValue(N, 1), Res.getValue(1));
+ unsigned ChainOp = N->getNumValues() - 1;
+ ReplaceValueWith(SDValue(N, ChainOp), Res.getValue(ChainOp));
return Res;
}
ReplaceValueWith(SDValue(N, 1), Tmp.second);
break;
}
+ case ISD::ATOMIC_CMP_SWAP_WITH_SUCCESS: {
+ AtomicSDNode *AN = cast<AtomicSDNode>(N);
+ SDVTList VTs = DAG.getVTList(N->getValueType(0), MVT::Other);
+ SDValue Tmp = DAG.getAtomicCmpSwap(
+ ISD::ATOMIC_CMP_SWAP, SDLoc(N), AN->getMemoryVT(), VTs,
+ N->getOperand(0), N->getOperand(1), N->getOperand(2), N->getOperand(3),
+ AN->getMemOperand(), AN->getSuccessOrdering(), AN->getFailureOrdering(),
+ AN->getSynchScope());
+
+ // Expanding to the strong ATOMIC_CMP_SWAP node means we can determine
+ // success simply by comparing the loaded value against the ingoing
+ // comparison.
+ SDValue Success = DAG.getSetCC(SDLoc(N), N->getValueType(1), Tmp,
+ N->getOperand(2), ISD::SETEQ);
+
+ SplitInteger(Tmp, Lo, Hi);
+ ReplaceValueWith(SDValue(N, 1), Success);
+ ReplaceValueWith(SDValue(N, 2), Tmp.getValue(1));
+ break;
+ }
case ISD::AND:
case ISD::OR:
SDValue &Lo, SDValue &Hi) {
SDLoc dl(N);
EVT VT = cast<AtomicSDNode>(N)->getMemoryVT();
+ SDVTList VTs = DAG.getVTList(VT, MVT::i1, MVT::Other);
SDValue Zero = DAG.getConstant(0, VT);
- SDValue Swap = DAG.getAtomic(ISD::ATOMIC_CMP_SWAP, dl, VT,
- N->getOperand(0),
- N->getOperand(1), Zero, Zero,
- cast<AtomicSDNode>(N)->getMemOperand(),
- cast<AtomicSDNode>(N)->getOrdering(),
- cast<AtomicSDNode>(N)->getOrdering(),
- cast<AtomicSDNode>(N)->getSynchScope());
+ SDValue Swap = DAG.getAtomicCmpSwap(
+ ISD::ATOMIC_CMP_SWAP_WITH_SUCCESS, dl,
+ cast<AtomicSDNode>(N)->getMemoryVT(), VTs, N->getOperand(0),
+ N->getOperand(1), Zero, Zero, cast<AtomicSDNode>(N)->getMemOperand(),
+ cast<AtomicSDNode>(N)->getOrdering(),
+ cast<AtomicSDNode>(N)->getOrdering(),
+ cast<AtomicSDNode>(N)->getSynchScope());
+
ReplaceValueWith(SDValue(N, 0), Swap.getValue(0));
- ReplaceValueWith(SDValue(N, 1), Swap.getValue(1));
+ ReplaceValueWith(SDValue(N, 1), Swap.getValue(2));
}
//===----------------------------------------------------------------------===//
SDValue PromoteIntRes_AssertZext(SDNode *N);
SDValue PromoteIntRes_Atomic0(AtomicSDNode *N);
SDValue PromoteIntRes_Atomic1(AtomicSDNode *N);
- SDValue PromoteIntRes_Atomic2(AtomicSDNode *N);
+ SDValue PromoteIntRes_AtomicCmpSwap(AtomicSDNode *N, unsigned ResNo);
SDValue PromoteIntRes_EXTRACT_SUBVECTOR(SDNode *N);
SDValue PromoteIntRes_VECTOR_SHUFFLE(SDNode *N);
SDValue PromoteIntRes_BUILD_VECTOR(SDNode *N);
break;
}
case ISD::ATOMIC_CMP_SWAP:
+ case ISD::ATOMIC_CMP_SWAP_WITH_SUCCESS:
case ISD::ATOMIC_SWAP:
case ISD::ATOMIC_LOAD_ADD:
case ISD::ATOMIC_LOAD_SUB:
Ordering, SynchScope);
}
-SDValue SelectionDAG::getAtomic(unsigned Opcode, SDLoc dl, EVT MemVT,
- SDValue Chain, SDValue Ptr, SDValue Cmp,
- SDValue Swp, MachinePointerInfo PtrInfo,
- unsigned Alignment,
- AtomicOrdering SuccessOrdering,
- AtomicOrdering FailureOrdering,
- SynchronizationScope SynchScope) {
+SDValue SelectionDAG::getAtomicCmpSwap(
+ unsigned Opcode, SDLoc dl, EVT MemVT, SDVTList VTs, SDValue Chain,
+ SDValue Ptr, SDValue Cmp, SDValue Swp, MachinePointerInfo PtrInfo,
+ unsigned Alignment, AtomicOrdering SuccessOrdering,
+ AtomicOrdering FailureOrdering, SynchronizationScope SynchScope) {
+ assert(Opcode == ISD::ATOMIC_CMP_SWAP ||
+ Opcode == ISD::ATOMIC_CMP_SWAP_WITH_SUCCESS);
+ assert(Cmp.getValueType() == Swp.getValueType() && "Invalid Atomic Op Types");
+
if (Alignment == 0) // Ensure that codegen never sees alignment 0
Alignment = getEVTAlignment(MemVT);
MachineFunction &MF = getMachineFunction();
- // All atomics are load and store, except for ATMOIC_LOAD and ATOMIC_STORE.
- // For now, atomics are considered to be volatile always.
// FIXME: Volatile isn't really correct; we should keep track of atomic
// orderings in the memoperand.
unsigned Flags = MachineMemOperand::MOVolatile;
- if (Opcode != ISD::ATOMIC_STORE)
- Flags |= MachineMemOperand::MOLoad;
- if (Opcode != ISD::ATOMIC_LOAD)
- Flags |= MachineMemOperand::MOStore;
+ Flags |= MachineMemOperand::MOLoad;
+ Flags |= MachineMemOperand::MOStore;
MachineMemOperand *MMO =
MF.getMachineMemOperand(PtrInfo, Flags, MemVT.getStoreSize(), Alignment);
- return getAtomic(Opcode, dl, MemVT, Chain, Ptr, Cmp, Swp, MMO,
- SuccessOrdering, FailureOrdering, SynchScope);
+ return getAtomicCmpSwap(Opcode, dl, MemVT, VTs, Chain, Ptr, Cmp, Swp, MMO,
+ SuccessOrdering, FailureOrdering, SynchScope);
}
-SDValue SelectionDAG::getAtomic(unsigned Opcode, SDLoc dl, EVT MemVT,
- SDValue Chain,
- SDValue Ptr, SDValue Cmp,
- SDValue Swp, MachineMemOperand *MMO,
- AtomicOrdering SuccessOrdering,
- AtomicOrdering FailureOrdering,
- SynchronizationScope SynchScope) {
- assert(Opcode == ISD::ATOMIC_CMP_SWAP && "Invalid Atomic Op");
+SDValue SelectionDAG::getAtomicCmpSwap(unsigned Opcode, SDLoc dl, EVT MemVT,
+ SDVTList VTs, SDValue Chain, SDValue Ptr,
+ SDValue Cmp, SDValue Swp,
+ MachineMemOperand *MMO,
+ AtomicOrdering SuccessOrdering,
+ AtomicOrdering FailureOrdering,
+ SynchronizationScope SynchScope) {
+ assert(Opcode == ISD::ATOMIC_CMP_SWAP ||
+ Opcode == ISD::ATOMIC_CMP_SWAP_WITH_SUCCESS);
assert(Cmp.getValueType() == Swp.getValueType() && "Invalid Atomic Op Types");
- EVT VT = Cmp.getValueType();
-
- SDVTList VTs = getVTList(VT, MVT::Other);
SDValue Ops[] = {Chain, Ptr, Cmp, Swp};
- return getAtomic(Opcode, dl, MemVT, VTs, Ops, MMO, SuccessOrdering,
- FailureOrdering, SynchScope);
+ return getAtomic(Opcode, dl, MemVT, VTs, Ops, MMO,
+ SuccessOrdering, FailureOrdering, SynchScope);
}
SDValue SelectionDAG::getAtomic(unsigned Opcode, SDLoc dl, EVT MemVT,
InChain = InsertFenceForAtomic(InChain, SuccessOrder, Scope, true, dl,
DAG, *TLI);
- SDValue L =
- DAG.getAtomic(ISD::ATOMIC_CMP_SWAP, dl,
- getValue(I.getCompareOperand()).getSimpleValueType(),
- InChain,
- getValue(I.getPointerOperand()),
- getValue(I.getCompareOperand()),
- getValue(I.getNewValOperand()),
- MachinePointerInfo(I.getPointerOperand()), 0 /* Alignment */,
- TLI->getInsertFencesForAtomic() ? Monotonic : SuccessOrder,
- TLI->getInsertFencesForAtomic() ? Monotonic : FailureOrder,
- Scope);
-
- SDValue OutChain = L.getValue(1);
+ MVT MemVT = getValue(I.getCompareOperand()).getSimpleValueType();
+ SDVTList VTs = DAG.getVTList(MemVT, MVT::i1, MVT::Other);
+ SDValue L = DAG.getAtomicCmpSwap(
+ ISD::ATOMIC_CMP_SWAP_WITH_SUCCESS, dl, MemVT, VTs, InChain,
+ getValue(I.getPointerOperand()), getValue(I.getCompareOperand()),
+ getValue(I.getNewValOperand()), MachinePointerInfo(I.getPointerOperand()),
+ 0 /* Alignment */,
+ TLI->getInsertFencesForAtomic() ? Monotonic : SuccessOrder,
+ TLI->getInsertFencesForAtomic() ? Monotonic : FailureOrder, Scope);
+
+ SDValue OutChain = L.getValue(2);
if (TLI->getInsertFencesForAtomic())
OutChain = InsertFenceForAtomic(OutChain, SuccessOrder, Scope, false, dl,
case ISD::PREFETCH: return "Prefetch";
case ISD::ATOMIC_FENCE: return "AtomicFence";
case ISD::ATOMIC_CMP_SWAP: return "AtomicCmpSwap";
+ case ISD::ATOMIC_CMP_SWAP_WITH_SUCCESS: return "AtomicCmpSwapWithSuccess";
case ISD::ATOMIC_SWAP: return "AtomicSwap";
case ISD::ATOMIC_LOAD_ADD: return "AtomicLoadAdd";
case ISD::ATOMIC_LOAD_SUB: return "AtomicLoadSub";
setIndexedStoreAction(IM, (MVT::SimpleValueType)VT, Expand);
}
+ // Most backends expect to see the node which just returns the value loaded.
+ setOperationAction(ISD::ATOMIC_CMP_SWAP_WITH_SUCCESS,
+ (MVT::SimpleValueType)VT, Expand);
+
// These operations default to expand.
setOperationAction(ISD::FGETSIGN, (MVT::SimpleValueType)VT, Expand);
setOperationAction(ISD::CONCAT_VECTORS, (MVT::SimpleValueType)VT, Expand);
(isa<StoreInst>(I) && cast<StoreInst>(I).isAtomic()))
Out << " atomic";
+ if (isa<AtomicCmpXchgInst>(I) && cast<AtomicCmpXchgInst>(I).isWeak())
+ Out << " weak";
+
// If this is a volatile operation, print out the volatile marker.
if ((isa<LoadInst>(I) && cast<LoadInst>(I).isVolatile()) ||
(isa<StoreInst>(I) && cast<StoreInst>(I).isVolatile()) ||
FI->getSynchScope() == cast<FenceInst>(I2)->getSynchScope();
if (const AtomicCmpXchgInst *CXI = dyn_cast<AtomicCmpXchgInst>(I1))
return CXI->isVolatile() == cast<AtomicCmpXchgInst>(I2)->isVolatile() &&
+ CXI->isWeak() == cast<AtomicCmpXchgInst>(I2)->isWeak() &&
CXI->getSuccessOrdering() ==
cast<AtomicCmpXchgInst>(I2)->getSuccessOrdering() &&
CXI->getFailureOrdering() ==
AtomicOrdering FailureOrdering,
SynchronizationScope SynchScope,
Instruction *InsertBefore)
- : Instruction(Cmp->getType(), AtomicCmpXchg,
- OperandTraits<AtomicCmpXchgInst>::op_begin(this),
- OperandTraits<AtomicCmpXchgInst>::operands(this),
- InsertBefore) {
+ : Instruction(
+ StructType::get(Cmp->getType(), Type::getInt1Ty(Cmp->getContext()),
+ nullptr),
+ AtomicCmpXchg, OperandTraits<AtomicCmpXchgInst>::op_begin(this),
+ OperandTraits<AtomicCmpXchgInst>::operands(this), InsertBefore) {
Init(Ptr, Cmp, NewVal, SuccessOrdering, FailureOrdering, SynchScope);
}
AtomicOrdering FailureOrdering,
SynchronizationScope SynchScope,
BasicBlock *InsertAtEnd)
- : Instruction(Cmp->getType(), AtomicCmpXchg,
- OperandTraits<AtomicCmpXchgInst>::op_begin(this),
- OperandTraits<AtomicCmpXchgInst>::operands(this),
- InsertAtEnd) {
+ : Instruction(
+ StructType::get(Cmp->getType(), Type::getInt1Ty(Cmp->getContext()),
+ nullptr),
+ AtomicCmpXchg, OperandTraits<AtomicCmpXchgInst>::op_begin(this),
+ OperandTraits<AtomicCmpXchgInst>::operands(this), InsertAtEnd) {
Init(Ptr, Cmp, NewVal, SuccessOrdering, FailureOrdering, SynchScope);
}
-
+
//===----------------------------------------------------------------------===//
// AtomicRMWInst Implementation
//===----------------------------------------------------------------------===//
getSuccessOrdering(), getFailureOrdering(),
getSynchScope());
Result->setVolatile(isVolatile());
+ Result->setWeak(isWeak());
return Result;
}
Out << "\");";
nl(Out) << iName << "->setVolatile("
<< (cxi->isVolatile() ? "true" : "false") << ");";
+ nl(Out) << iName << "->setWeak("
+ << (cxi->isWeak() ? "true" : "false") << ");";
break;
}
case Instruction::AtomicRMW: {
// (The only way to get a 16-byte load is cmpxchg16b)
// FIXME: 16-byte ATOMIC_CMP_SWAP isn't actually hooked up at the moment.
SDValue Zero = DAG.getConstant(0, VT);
- SDValue Swap = DAG.getAtomic(ISD::ATOMIC_CMP_SWAP, dl, VT,
- Node->getOperand(0),
- Node->getOperand(1), Zero, Zero,
- cast<AtomicSDNode>(Node)->getMemOperand(),
- cast<AtomicSDNode>(Node)->getOrdering(),
- cast<AtomicSDNode>(Node)->getOrdering(),
- cast<AtomicSDNode>(Node)->getSynchScope());
+ SDVTList VTs = DAG.getVTList(VT, MVT::Other);
+ SDValue Swap =
+ DAG.getAtomicCmpSwap(ISD::ATOMIC_CMP_SWAP, dl, VT, VTs,
+ Node->getOperand(0), Node->getOperand(1), Zero, Zero,
+ cast<AtomicSDNode>(Node)->getMemOperand(),
+ cast<AtomicSDNode>(Node)->getOrdering(),
+ cast<AtomicSDNode>(Node)->getOrdering(),
+ cast<AtomicSDNode>(Node)->getSynchScope());
Results.push_back(Swap.getValue(0));
Results.push_back(Swap.getValue(1));
}
if (int Res = cmpNumbers(CXI->isVolatile(),
cast<AtomicCmpXchgInst>(R)->isVolatile()))
return Res;
+ if (int Res = cmpNumbers(CXI->isWeak(),
+ cast<AtomicCmpXchgInst>(R)->isWeak()))
+ return Res;
if (int Res = cmpNumbers(CXI->getSuccessOrdering(),
cast<AtomicCmpXchgInst>(R)->getSuccessOrdering()))
return Res;
IRB.CreateIntCast(CASI->getNewValOperand(), Ty, false),
createOrdering(&IRB, CASI->getSuccessOrdering()),
createOrdering(&IRB, CASI->getFailureOrdering())};
- CallInst *C = CallInst::Create(TsanAtomicCAS[Idx], ArrayRef<Value*>(Args));
- ReplaceInstWithInst(I, C);
+ CallInst *C = IRB.CreateCall(TsanAtomicCAS[Idx], Args);
+ Value *Success = IRB.CreateICmpEQ(C, CASI->getCompareOperand());
+
+ Value *Res = IRB.CreateInsertValue(UndefValue::get(CASI->getType()), C, 0);
+ Res = IRB.CreateInsertValue(Res, Success, 1);
+
+ I->replaceAllUsesWith(Res);
+ I->eraseFromParent();
} else if (FenceInst *FI = dyn_cast<FenceInst>(I)) {
Value *Args[] = {createOrdering(&IRB, FI->getOrdering())};
Function *F = FI->getSynchScope() == SingleThread ?
Value *Res = Builder.CreateSelect(Equal, Val, Orig);
Builder.CreateStore(Res, Ptr);
- CXI->replaceAllUsesWith(Orig);
+ Res = Builder.CreateInsertValue(UndefValue::get(CXI->getType()), Orig, 0);
+ Res = Builder.CreateInsertValue(Res, Equal, 1);
+
+ CXI->replaceAllUsesWith(Res);
CXI->eraseFromParent();
return true;
}
cmpxchg volatile i32* %x, i32 0, i32 1 acq_rel acquire
; CHECK: cmpxchg i32* %x, i32 42, i32 0 acq_rel monotonic
cmpxchg i32* %x, i32 42, i32 0 acq_rel monotonic
+ ; CHECK: cmpxchg weak i32* %x, i32 13, i32 0 seq_cst monotonic
+ cmpxchg weak i32* %x, i32 13, i32 0 seq_cst monotonic
; CHECK: atomicrmw add i32* %x, i32 10 seq_cst
atomicrmw add i32* %x, i32 10 seq_cst
; CHECK: atomicrmw volatile xchg i32* %x, i32 10 monotonic
--- /dev/null
+; RUN: llvm-as %s -o - | llvm-dis | FileCheck %s
+
+define void @test_cmpxchg(i32* %addr, i32 %desired, i32 %new) {
+ cmpxchg i32* %addr, i32 %desired, i32 %new seq_cst seq_cst
+ ; CHECK: cmpxchg i32* %addr, i32 %desired, i32 %new seq_cst seq_cst
+
+ cmpxchg volatile i32* %addr, i32 %desired, i32 %new seq_cst monotonic
+ ; CHECK: cmpxchg volatile i32* %addr, i32 %desired, i32 %new seq_cst monotonic
+
+ cmpxchg weak i32* %addr, i32 %desired, i32 %new acq_rel acquire
+ ; CHECK: cmpxchg weak i32* %addr, i32 %desired, i32 %new acq_rel acquire
+
+ cmpxchg weak volatile i32* %addr, i32 %desired, i32 %new singlethread release monotonic
+ ; CHECK: cmpxchg weak volatile i32* %addr, i32 %desired, i32 %new singlethread release monotonic
+
+ ret void
+}
\ No newline at end of file
entry:\r
;cmpxchg [volatile] <ty>* <pointer>, <ty> <cmp>, <ty> <new> [singlethread] <ordering>\r
\r
-; CHECK: %res1 = cmpxchg i32* %ptr, i32 %cmp, i32 %new monotonic monotonic\r
+; CHECK: [[TMP:%[a-z0-9]+]] = cmpxchg i32* %ptr, i32 %cmp, i32 %new monotonic monotonic\r
+; CHECK-NEXT: %res1 = extractvalue { i32, i1 } [[TMP]], 0\r
%res1 = cmpxchg i32* %ptr, i32 %cmp, i32 %new monotonic monotonic\r
\r
-; CHECK-NEXT: %res2 = cmpxchg volatile i32* %ptr, i32 %cmp, i32 %new monotonic monotonic\r
+; CHECK-NEXT: [[TMP:%[a-z0-9]+]] = cmpxchg volatile i32* %ptr, i32 %cmp, i32 %new monotonic monotonic\r
+; CHECK-NEXT: %res2 = extractvalue { i32, i1 } [[TMP]], 0\r
%res2 = cmpxchg volatile i32* %ptr, i32 %cmp, i32 %new monotonic monotonic\r
\r
-; CHECK-NEXT: %res3 = cmpxchg i32* %ptr, i32 %cmp, i32 %new singlethread monotonic monotonic\r
+; CHECK-NEXT: [[TMP:%[a-z0-9]+]] = cmpxchg i32* %ptr, i32 %cmp, i32 %new singlethread monotonic monotonic\r
+; CHECK-NEXT: %res3 = extractvalue { i32, i1 } [[TMP]], 0\r
%res3 = cmpxchg i32* %ptr, i32 %cmp, i32 %new singlethread monotonic monotonic\r
\r
-; CHECK-NEXT: %res4 = cmpxchg volatile i32* %ptr, i32 %cmp, i32 %new singlethread monotonic monotonic\r
+; CHECK-NEXT: [[TMP:%[a-z0-9]+]] = cmpxchg volatile i32* %ptr, i32 %cmp, i32 %new singlethread monotonic monotonic\r
+; CHECK-NEXT: %res4 = extractvalue { i32, i1 } [[TMP]], 0\r
%res4 = cmpxchg volatile i32* %ptr, i32 %cmp, i32 %new singlethread monotonic monotonic\r
\r
\r
-; CHECK-NEXT: %res5 = cmpxchg i32* %ptr, i32 %cmp, i32 %new acquire acquire\r
+; CHECK-NEXT: [[TMP:%[a-z0-9]+]] = cmpxchg i32* %ptr, i32 %cmp, i32 %new acquire acquire\r
+; CHECK-NEXT: %res5 = extractvalue { i32, i1 } [[TMP]], 0\r
%res5 = cmpxchg i32* %ptr, i32 %cmp, i32 %new acquire acquire\r
\r
-; CHECK-NEXT: %res6 = cmpxchg volatile i32* %ptr, i32 %cmp, i32 %new acquire acquire\r
+; CHECK-NEXT: [[TMP:%[a-z0-9]+]] = cmpxchg volatile i32* %ptr, i32 %cmp, i32 %new acquire acquire\r
+; CHECK-NEXT: %res6 = extractvalue { i32, i1 } [[TMP]], 0\r
%res6 = cmpxchg volatile i32* %ptr, i32 %cmp, i32 %new acquire acquire\r
\r
-; CHECK-NEXT: %res7 = cmpxchg i32* %ptr, i32 %cmp, i32 %new singlethread acquire acquire\r
+; CHECK-NEXT: [[TMP:%[a-z0-9]+]] = cmpxchg i32* %ptr, i32 %cmp, i32 %new singlethread acquire acquire\r
+; CHECK-NEXT: %res7 = extractvalue { i32, i1 } [[TMP]], 0\r
%res7 = cmpxchg i32* %ptr, i32 %cmp, i32 %new singlethread acquire acquire\r
\r
-; CHECK-NEXT: %res8 = cmpxchg volatile i32* %ptr, i32 %cmp, i32 %new singlethread acquire acquire\r
+; CHECK-NEXT: [[TMP:%[a-z0-9]+]] = cmpxchg volatile i32* %ptr, i32 %cmp, i32 %new singlethread acquire acquire\r
+; CHECK-NEXT: %res8 = extractvalue { i32, i1 } [[TMP]], 0\r
%res8 = cmpxchg volatile i32* %ptr, i32 %cmp, i32 %new singlethread acquire acquire\r
\r
\r
-; CHECK-NEXT: %res9 = cmpxchg i32* %ptr, i32 %cmp, i32 %new release monotonic\r
+; CHECK-NEXT: [[TMP:%[a-z0-9]+]] = cmpxchg i32* %ptr, i32 %cmp, i32 %new release monotonic\r
+; CHECK-NEXT: %res9 = extractvalue { i32, i1 } [[TMP]], 0\r
%res9 = cmpxchg i32* %ptr, i32 %cmp, i32 %new release monotonic\r
\r
-; CHECK-NEXT: %res10 = cmpxchg volatile i32* %ptr, i32 %cmp, i32 %new release monotonic\r
+; CHECK-NEXT: [[TMP:%[a-z0-9]+]] = cmpxchg volatile i32* %ptr, i32 %cmp, i32 %new release monotonic\r
+; CHECK-NEXT: %res10 = extractvalue { i32, i1 } [[TMP]], 0\r
%res10 = cmpxchg volatile i32* %ptr, i32 %cmp, i32 %new release monotonic\r
\r
-; CHECK-NEXT: %res11 = cmpxchg i32* %ptr, i32 %cmp, i32 %new singlethread release monotonic\r
+; CHECK-NEXT: [[TMP:%[a-z0-9]+]] = cmpxchg i32* %ptr, i32 %cmp, i32 %new singlethread release monotonic\r
+; CHECK-NEXT: %res11 = extractvalue { i32, i1 } [[TMP]], 0\r
%res11 = cmpxchg i32* %ptr, i32 %cmp, i32 %new singlethread release monotonic\r
\r
-; CHECK-NEXT: %res12 = cmpxchg volatile i32* %ptr, i32 %cmp, i32 %new singlethread release monotonic\r
+; CHECK-NEXT: [[TMP:%[a-z0-9]+]] = cmpxchg volatile i32* %ptr, i32 %cmp, i32 %new singlethread release monotonic\r
+; CHECK-NEXT: %res12 = extractvalue { i32, i1 } [[TMP]], 0\r
%res12 = cmpxchg volatile i32* %ptr, i32 %cmp, i32 %new singlethread release monotonic\r
\r
\r
-; CHECK-NEXT: %res13 = cmpxchg i32* %ptr, i32 %cmp, i32 %new acq_rel acquire\r
+; CHECK-NEXT: [[TMP:%[a-z0-9]+]] = cmpxchg i32* %ptr, i32 %cmp, i32 %new acq_rel acquire\r
+; CHECK-NEXT: %res13 = extractvalue { i32, i1 } [[TMP]], 0\r
%res13 = cmpxchg i32* %ptr, i32 %cmp, i32 %new acq_rel acquire\r
\r
-; CHECK-NEXT: %res14 = cmpxchg volatile i32* %ptr, i32 %cmp, i32 %new acq_rel acquire\r
+; CHECK-NEXT: [[TMP:%[a-z0-9]+]] = cmpxchg volatile i32* %ptr, i32 %cmp, i32 %new acq_rel acquire\r
+; CHECK-NEXT: %res14 = extractvalue { i32, i1 } [[TMP]], 0\r
%res14 = cmpxchg volatile i32* %ptr, i32 %cmp, i32 %new acq_rel acquire\r
\r
-; CHECK-NEXT: %res15 = cmpxchg i32* %ptr, i32 %cmp, i32 %new singlethread acq_rel acquire\r
+; CHECK-NEXT: [[TMP:%[a-z0-9]+]] = cmpxchg i32* %ptr, i32 %cmp, i32 %new singlethread acq_rel acquire\r
+; CHECK-NEXT: %res15 = extractvalue { i32, i1 } [[TMP]], 0\r
%res15 = cmpxchg i32* %ptr, i32 %cmp, i32 %new singlethread acq_rel acquire\r
\r
-; CHECK-NEXT: %res16 = cmpxchg volatile i32* %ptr, i32 %cmp, i32 %new singlethread acq_rel acquire\r
+; CHECK-NEXT: [[TMP:%[a-z0-9]+]] = cmpxchg volatile i32* %ptr, i32 %cmp, i32 %new singlethread acq_rel acquire\r
+; CHECK-NEXT: %res16 = extractvalue { i32, i1 } [[TMP]], 0\r
%res16 = cmpxchg volatile i32* %ptr, i32 %cmp, i32 %new singlethread acq_rel acquire\r
\r
\r
-; CHECK-NEXT: %res17 = cmpxchg i32* %ptr, i32 %cmp, i32 %new seq_cst seq_cst\r
+; CHECK-NEXT: [[TMP:%[a-z0-9]+]] = cmpxchg i32* %ptr, i32 %cmp, i32 %new seq_cst seq_cst\r
+; CHECK-NEXT: %res17 = extractvalue { i32, i1 } [[TMP]], 0\r
%res17 = cmpxchg i32* %ptr, i32 %cmp, i32 %new seq_cst seq_cst\r
\r
-; CHECK-NEXT: %res18 = cmpxchg volatile i32* %ptr, i32 %cmp, i32 %new seq_cst seq_cst\r
+; CHECK-NEXT: [[TMP:%[a-z0-9]+]] = cmpxchg volatile i32* %ptr, i32 %cmp, i32 %new seq_cst seq_cst\r
+; CHECK-NEXT: %res18 = extractvalue { i32, i1 } [[TMP]], 0\r
%res18 = cmpxchg volatile i32* %ptr, i32 %cmp, i32 %new seq_cst seq_cst\r
\r
-; CHECK-NEXT: %res19 = cmpxchg i32* %ptr, i32 %cmp, i32 %new singlethread seq_cst seq_cst\r
+; CHECK-NEXT: [[TMP:%[a-z0-9]+]] = cmpxchg i32* %ptr, i32 %cmp, i32 %new singlethread seq_cst seq_cst\r
+; CHECK-NEXT: %res19 = extractvalue { i32, i1 } [[TMP]], 0\r
%res19 = cmpxchg i32* %ptr, i32 %cmp, i32 %new singlethread seq_cst seq_cst\r
\r
-; CHECK-NEXT: %res20 = cmpxchg volatile i32* %ptr, i32 %cmp, i32 %new singlethread seq_cst seq_cst\r
+; CHECK-NEXT: [[TMP:%[a-z0-9]+]] = cmpxchg volatile i32* %ptr, i32 %cmp, i32 %new singlethread seq_cst seq_cst\r
+; CHECK-NEXT: %res20 = extractvalue { i32, i1 } [[TMP]], 0\r
%res20 = cmpxchg volatile i32* %ptr, i32 %cmp, i32 %new singlethread seq_cst seq_cst\r
\r
ret void\r
--- /dev/null
+; RUN: llvm-dis < %s.bc | FileCheck %s
+
+; cmpxchg-upgrade.ll.bc was produced by running a version of llvm-as from just
+; before the IR change on this file.
+
+define i32 @test(i32* %addr, i32 %old, i32 %new) {
+; CHECK: [[TMP:%.*]] = cmpxchg i32* %addr, i32 %old, i32 %new seq_cst monotonic
+; CHECK: %val = extractvalue { i32, i1 } [[TMP]], 0
+ %val = cmpxchg i32* %addr, i32 %old, i32 %new seq_cst monotonic
+ ret i32 %val
+}
+
+define i32 @test(i32* %addr, i32 %old, i32 %new) {
+ ret i1 %val
+}
; CHECK: stxp [[SCRATCH_RES:w[0-9]+]], x4, x5, [x[[ADDR]]]
; CHECK: cbnz [[SCRATCH_RES]], [[LABEL]]
; CHECK: [[DONE]]:
- %val = cmpxchg i128* %p, i128 %oldval, i128 %newval acquire acquire
+ %pair = cmpxchg i128* %p, i128 %oldval, i128 %newval acquire acquire
+ %val = extractvalue { i128, i1 } %pair, 0
ret i128 %val
}
; CHECK: stxr [[SCRATCH_REG:w[0-9]+]], [[NEWVAL_REG]], [x0]
; CHECK: cbnz [[SCRATCH_REG]], [[LABEL]]
; CHECK: [[LABEL2]]:
- %val = cmpxchg i32* %p, i32 7, i32 4 acquire acquire
+ %pair = cmpxchg i32* %p, i32 7, i32 4 acquire acquire
+ %val = extractvalue { i32, i1 } %pair, 0
ret i32 %val
}
; CHECK: stxr [[SCRATCH_REG:w[0-9]+]], x[[NEWVAL_REG]], [x0]
; CHECK: cbnz [[SCRATCH_REG]], [[LABEL]]
; CHECK: [[LABEL2]]:
- %val = cmpxchg i64* %p, i64 7, i64 4 monotonic monotonic
+ %pair = cmpxchg i64* %p, i64 7, i64 4 monotonic monotonic
+ %val = extractvalue { i64, i1 } %pair, 0
ret i64 %val
}
define i8 @test_atomic_cmpxchg_i8(i8 %wanted, i8 %new) nounwind {
; CHECK-LABEL: test_atomic_cmpxchg_i8:
- %old = cmpxchg i8* @var8, i8 %wanted, i8 %new acquire acquire
+ %pair = cmpxchg i8* @var8, i8 %wanted, i8 %new acquire acquire
+ %old = extractvalue { i8, i1 } %pair, 0
+
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var8
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var8
define i16 @test_atomic_cmpxchg_i16(i16 %wanted, i16 %new) nounwind {
; CHECK-LABEL: test_atomic_cmpxchg_i16:
- %old = cmpxchg i16* @var16, i16 %wanted, i16 %new seq_cst seq_cst
+ %pair = cmpxchg i16* @var16, i16 %wanted, i16 %new seq_cst seq_cst
+ %old = extractvalue { i16, i1 } %pair, 0
+
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var16
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var16
define i32 @test_atomic_cmpxchg_i32(i32 %wanted, i32 %new) nounwind {
; CHECK-LABEL: test_atomic_cmpxchg_i32:
- %old = cmpxchg i32* @var32, i32 %wanted, i32 %new release monotonic
+ %pair = cmpxchg i32* @var32, i32 %wanted, i32 %new release monotonic
+ %old = extractvalue { i32, i1 } %pair, 0
+
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
define void @test_atomic_cmpxchg_i64(i64 %wanted, i64 %new) nounwind {
; CHECK-LABEL: test_atomic_cmpxchg_i64:
- %old = cmpxchg i64* @var64, i64 %wanted, i64 %new monotonic monotonic
+ %pair = cmpxchg i64* @var64, i64 %wanted, i64 %new monotonic monotonic
+ %old = extractvalue { i64, i1 } %pair, 0
+
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: mov w0, wzr
; CHECK: ret
- %loaded = cmpxchg i32* %p, i32 %oldval, i32 %newval seq_cst seq_cst
- %success = icmp eq i32 %loaded, %oldval
+ %pair = cmpxchg i32* %p, i32 %oldval, i32 %newval seq_cst seq_cst
+ %success = extractvalue { i32, i1 } %pair, 1
%conv = zext i1 %success to i32
ret i32 %conv
}
; CHECK: cbnz [[STATUS]], [[LOOP]]
; CHECK-NOT: cmp {{w[0-9]+}}, {{w[0-9]+}}
-; CHECK: mov w0, wzr
+ ; FIXME: DAG combine should be able to deal with this.
+; CHECK: orr [[TMP:w[0-9]+]], wzr, #0x1
+; CHECK: eor w0, [[TMP]], #0x1
; CHECK: ret
; CHECK: [[FAILED]]:
; CHECK-NOT: cmp {{w[0-9]+}}, {{w[0-9]+}}
-; CHECK: orr w0, wzr, #0x1
+; CHECK: mov [[TMP:w[0-9]+]], wzr
+; CHECK: eor w0, [[TMP]], #0x1
; CHECK: ret
- %loaded = cmpxchg i8* %value, i8 %oldValue, i8 %newValue acq_rel monotonic
- %failure = icmp ne i8 %loaded, %oldValue
+ %pair = cmpxchg i8* %value, i8 %oldValue, i8 %newValue acq_rel monotonic
+ %success = extractvalue { i8, i1 } %pair, 1
+ %failure = xor i1 %success, 1
ret i1 %failure
}
; CHECK-NOT: cmp {{w[0-9]+}}, {{w[0-9]+}}
; CHECK: b _baz
- %loaded = cmpxchg i32* %p, i32 %oldval, i32 %newval seq_cst seq_cst
- %success = icmp eq i32 %loaded, %oldval
+ %pair = cmpxchg i32* %p, i32 %oldval, i32 %newval seq_cst seq_cst
+ %success = extractvalue { i32, i1 } %pair, 1
br i1 %success, label %true, label %false
true:
; CHECK-THUMB: bne
; CHECK-THUMB: dmb {{ish$}}
- %r = cmpxchg i64* %ptr, i64 %val1, i64 %val2 seq_cst seq_cst
+ %pair = cmpxchg i64* %ptr, i64 %val1, i64 %val2 seq_cst seq_cst
+ %r = extractvalue { i64, i1 } %pair, 0
ret i64 %r
}
; T2: ldrexb
; T2: strexb
%tmp0 = cmpxchg i8* %a, i8 %b, i8 %c monotonic monotonic
- ret i8 %tmp0
+ %tmp1 = extractvalue { i8, i1 } %tmp0, 0
+ ret i8 %tmp1
}
define i32 @test_cmpxchg_fail_order(i32 *%addr, i32 %desired, i32 %new) {
; CHECK-LABEL: test_cmpxchg_fail_order:
- %oldval = cmpxchg i32* %addr, i32 %desired, i32 %new seq_cst monotonic
+ %pair = cmpxchg i32* %addr, i32 %desired, i32 %new seq_cst monotonic
+ %oldval = extractvalue { i32, i1 } %pair, 0
; CHECK: dmb ish
; CHECK: [[LOOP_BB:\.?LBB[0-9]+_1]]:
; CHECK: ldrex [[OLDVAL:r[0-9]+]], [r[[ADDR:[0-9]+]]]
define i32 @test_cmpxchg_fail_order1(i32 *%addr, i32 %desired, i32 %new) {
; CHECK-LABEL: test_cmpxchg_fail_order1:
- %oldval = cmpxchg i32* %addr, i32 %desired, i32 %new acquire acquire
+ %pair = cmpxchg i32* %addr, i32 %desired, i32 %new acquire acquire
+ %oldval = extractvalue { i32, i1 } %pair, 0
; CHECK-NOT: dmb ish
; CHECK: [[LOOP_BB:\.?LBB[0-9]+_1]]:
; CHECK: ldrex [[OLDVAL:r[0-9]+]], [r[[ADDR:[0-9]+]]]
define i8 @test_atomic_cmpxchg_i8(i8 zeroext %wanted, i8 zeroext %new) nounwind {
; CHECK-LABEL: test_atomic_cmpxchg_i8:
- %old = cmpxchg i8* @var8, i8 %wanted, i8 %new acquire acquire
+ %pair = cmpxchg i8* @var8, i8 %wanted, i8 %new acquire acquire
+ %old = extractvalue { i8, i1 } %pair, 0
; CHECK-NOT: dmb
; CHECK-NOT: mcr
; CHECK: movw r[[ADDR:[0-9]+]], :lower16:var8
define i16 @test_atomic_cmpxchg_i16(i16 zeroext %wanted, i16 zeroext %new) nounwind {
; CHECK-LABEL: test_atomic_cmpxchg_i16:
- %old = cmpxchg i16* @var16, i16 %wanted, i16 %new seq_cst seq_cst
+ %pair = cmpxchg i16* @var16, i16 %wanted, i16 %new seq_cst seq_cst
+ %old = extractvalue { i16, i1 } %pair, 0
; CHECK-NOT: dmb
; CHECK-NOT: mcr
; CHECK: movw r[[ADDR:[0-9]+]], :lower16:var16
define void @test_atomic_cmpxchg_i32(i32 %wanted, i32 %new) nounwind {
; CHECK-LABEL: test_atomic_cmpxchg_i32:
- %old = cmpxchg i32* @var32, i32 %wanted, i32 %new release monotonic
+ %pair = cmpxchg i32* @var32, i32 %wanted, i32 %new release monotonic
+ %old = extractvalue { i32, i1 } %pair, 0
store i32 %old, i32* @var32
; CHECK-NOT: dmb
; CHECK-NOT: mcr
define void @test_atomic_cmpxchg_i64(i64 %wanted, i64 %new) nounwind {
; CHECK-LABEL: test_atomic_cmpxchg_i64:
- %old = cmpxchg i64* @var64, i64 %wanted, i64 %new monotonic monotonic
+ %pair = cmpxchg i64* @var64, i64 %wanted, i64 %new monotonic monotonic
+ %old = extractvalue { i64, i1 } %pair, 0
; CHECK-NOT: dmb
; CHECK-NOT: mcr
; CHECK: movw r[[ADDR:[0-9]+]], :lower16:var64
; CHECK: dmb ish
; CHECK: bx lr
- %loaded = cmpxchg i32* %p, i32 %oldval, i32 %newval seq_cst seq_cst
- %success = icmp eq i32 %loaded, %oldval
+ %pair = cmpxchg i32* %p, i32 %oldval, i32 %newval seq_cst seq_cst
+ %success = extractvalue { i32, i1 } %pair, 1
%conv = zext i1 %success to i32
ret i32 %conv
}
; CHECK: [[LOOP:LBB[0-9]+_[0-9]+]]:
; CHECK: ldrexb [[LOADED:r[0-9]+]], [r0]
; CHECK: cmp [[LOADED]], [[OLDBYTE]]
-
-; CHECK: itt ne
-; CHECK: movne r0, #1
-; CHECK: bxne lr
+; CHECK: bne [[FAIL:LBB[0-9]+_[0-9]+]]
; CHECK: strexb [[STATUS:r[0-9]+]], {{r[0-9]+}}, [r0]
; CHECK: cmp [[STATUS]], #0
; CHECK: bne [[LOOP]]
+ ; FIXME: this eor is redundant. Need to teach DAG combine that.
; CHECK-NOT: cmp {{r[0-9]+}}, {{r[0-9]+}}
-; CHECK: movs r0, #0
+; CHECK: movs [[TMP:r[0-9]+]], #1
+; CHECK: eor r0, [[TMP]], #1
; CHECK: bx lr
- %loaded = cmpxchg i8* %value, i8 %oldValue, i8 %newValue acq_rel monotonic
- %failure = icmp ne i8 %loaded, %oldValue
+; CHECK: [[FAIL]]:
+; CHECK: movs [[TMP:r[0-9]+]], #0
+; CHECK: eor r0, [[TMP]], #1
+; CHECK: bx lr
+
+
+ %pair = cmpxchg i8* %value, i8 %oldValue, i8 %newValue acq_rel monotonic
+ %success = extractvalue { i8, i1 } %pair, 1
+ %failure = xor i1 %success, 1
ret i1 %failure
}
; CHECK: dmb ish
; CHECK: b.w _baz
- %loaded = cmpxchg i32* %p, i32 %oldval, i32 %newval seq_cst seq_cst
- %success = icmp eq i32 %loaded, %oldval
+ %pair = cmpxchg i32* %p, i32 %oldval, i32 %newval seq_cst seq_cst
+ %success = extractvalue { i32, i1 } %pair, 1
br i1 %success, label %true, label %false
true:
; CHECK: AtomicCmpXchgInst* [[INST:[a-zA-Z0-9_]+]] = new AtomicCmpXchgInst({{.*}}, SequentiallyConsistent, Monotonic, CrossThread
; CHECK: [[INST]]->setName("inst0");
; CHECK: [[INST]]->setVolatile(false);
+ ; CHECK: [[INST]]->setWeak(false);
%inst1 = cmpxchg volatile i32* %addr, i32 %desired, i32 %new singlethread acq_rel acquire
; CHECK: AtomicCmpXchgInst* [[INST:[a-zA-Z0-9_]+]] = new AtomicCmpXchgInst({{.*}}, AcquireRelease, Acquire, SingleThread
; CHECK: [[INST]]->setName("inst1");
; CHECK: [[INST]]->setVolatile(true);
+ ; CHECK: [[INST]]->setWeak(false);
+
+ %inst2 = cmpxchg weak i32* %addr, i32 %desired, i32 %new seq_cst monotonic
+ ; CHECK: AtomicCmpXchgInst* [[INST:[a-zA-Z0-9_]+]] = new AtomicCmpXchgInst({{.*}}, SequentiallyConsistent, Monotonic, CrossThread
+ ; CHECK: [[INST]]->setName("inst2");
+ ; CHECK: [[INST]]->setVolatile(false);
+ ; CHECK: [[INST]]->setWeak(true);
+
+ %inst3 = cmpxchg weak volatile i32* %addr, i32 %desired, i32 %new singlethread acq_rel acquire
+ ; CHECK: AtomicCmpXchgInst* [[INST:[a-zA-Z0-9_]+]] = new AtomicCmpXchgInst({{.*}}, AcquireRelease, Acquire, SingleThread
+ ; CHECK: [[INST]]->setName("inst3");
+ ; CHECK: [[INST]]->setVolatile(true);
+ ; CHECK: [[INST]]->setWeak(true);
ret void
}
store i32 %newval, i32* %newval.addr, align 4
%tmp = load i32* %newval.addr, align 4
%0 = cmpxchg i32* @x, i32 %oldval, i32 %tmp monotonic monotonic
- ret i32 %0
+ %1 = extractvalue { i32, i1 } %0, 0
+ ret i32 %1
; CHECK-EL-LABEL: AtomicCmpSwap32:
; CHECK-EL: lw $[[R0:[0-9]+]], %got(x)
define signext i8 @AtomicCmpSwap8(i8 signext %oldval, i8 signext %newval) nounwind {
entry:
- %0 = cmpxchg i8* @y, i8 %oldval, i8 %newval monotonic monotonic
+ %pair0 = cmpxchg i8* @y, i8 %oldval, i8 %newval monotonic monotonic
+ %0 = extractvalue { i8, i1 } %pair0, 0
ret i8 %0
; CHECK-EL-LABEL: AtomicCmpSwap8:
define i32 @zeroreg() nounwind {
entry:
- %0 = cmpxchg i32* @a, i32 1, i32 0 seq_cst seq_cst
+ %pair0 = cmpxchg i32* @a, i32 1, i32 0 seq_cst seq_cst
+ %0 = extractvalue { i32, i1 } %pair0, 0
%1 = icmp eq i32 %0, 1
%conv = zext i1 %1 to i32
ret i32 %conv
%add.i = add nsw i32 %0, 2
%1 = load volatile i32* %x, align 4
%call1 = call i32 (i8*, ...)* @printf(i8* getelementptr inbounds ([8 x i8]* @.str, i32 0, i32 0), i32 %add.i, i32 %1) nounwind
- %2 = cmpxchg i32* %x, i32 1, i32 2 seq_cst seq_cst
+ %pair = cmpxchg i32* %x, i32 1, i32 2 seq_cst seq_cst
+ %2 = extractvalue { i32, i1 } %pair, 0
%3 = load volatile i32* %x, align 4
%call2 = call i32 (i8*, ...)* @printf(i8* getelementptr inbounds ([8 x i8]* @.str, i32 0, i32 0), i32 %2, i32 %3) nounwind
%4 = atomicrmw xchg i32* %x, i32 1 seq_cst
entry:
%0 = load i8* @uc, align 1
%1 = load i8* @sc, align 1
- %2 = cmpxchg i8* @sc, i8 %0, i8 %1 monotonic monotonic
+ %pair2 = cmpxchg i8* @sc, i8 %0, i8 %1 monotonic monotonic
+ %2 = extractvalue { i8, i1 } %pair2, 0
store i8 %2, i8* @sc, align 1
%3 = load i8* @uc, align 1
%4 = load i8* @sc, align 1
- %5 = cmpxchg i8* @uc, i8 %3, i8 %4 monotonic monotonic
+ %pair5 = cmpxchg i8* @uc, i8 %3, i8 %4 monotonic monotonic
+ %5 = extractvalue { i8, i1 } %pair5, 0
store i8 %5, i8* @uc, align 1
%6 = load i8* @uc, align 1
%7 = zext i8 %6 to i16
%8 = load i8* @sc, align 1
%9 = sext i8 %8 to i16
%10 = bitcast i8* bitcast (i16* @ss to i8*) to i16*
- %11 = cmpxchg i16* %10, i16 %7, i16 %9 monotonic monotonic
+ %pair11 = cmpxchg i16* %10, i16 %7, i16 %9 monotonic monotonic
+ %11 = extractvalue { i16, i1 } %pair11, 0
store i16 %11, i16* @ss, align 2
%12 = load i8* @uc, align 1
%13 = zext i8 %12 to i16
%14 = load i8* @sc, align 1
%15 = sext i8 %14 to i16
%16 = bitcast i8* bitcast (i16* @us to i8*) to i16*
- %17 = cmpxchg i16* %16, i16 %13, i16 %15 monotonic monotonic
+ %pair17 = cmpxchg i16* %16, i16 %13, i16 %15 monotonic monotonic
+ %17 = extractvalue { i16, i1 } %pair17, 0
store i16 %17, i16* @us, align 2
%18 = load i8* @uc, align 1
%19 = zext i8 %18 to i32
%20 = load i8* @sc, align 1
%21 = sext i8 %20 to i32
%22 = bitcast i8* bitcast (i32* @si to i8*) to i32*
- %23 = cmpxchg i32* %22, i32 %19, i32 %21 monotonic monotonic
+ %pair23 = cmpxchg i32* %22, i32 %19, i32 %21 monotonic monotonic
+ %23 = extractvalue { i32, i1 } %pair23, 0
store i32 %23, i32* @si, align 4
%24 = load i8* @uc, align 1
%25 = zext i8 %24 to i32
%26 = load i8* @sc, align 1
%27 = sext i8 %26 to i32
%28 = bitcast i8* bitcast (i32* @ui to i8*) to i32*
- %29 = cmpxchg i32* %28, i32 %25, i32 %27 monotonic monotonic
+ %pair29 = cmpxchg i32* %28, i32 %25, i32 %27 monotonic monotonic
+ %29 = extractvalue { i32, i1 } %pair29, 0
store i32 %29, i32* @ui, align 4
%30 = load i8* @uc, align 1
%31 = zext i8 %30 to i32
%32 = load i8* @sc, align 1
%33 = sext i8 %32 to i32
%34 = bitcast i8* bitcast (i32* @sl to i8*) to i32*
- %35 = cmpxchg i32* %34, i32 %31, i32 %33 monotonic monotonic
+ %pair35 = cmpxchg i32* %34, i32 %31, i32 %33 monotonic monotonic
+ %35 = extractvalue { i32, i1 } %pair35, 0
store i32 %35, i32* @sl, align 4
%36 = load i8* @uc, align 1
%37 = zext i8 %36 to i32
%38 = load i8* @sc, align 1
%39 = sext i8 %38 to i32
%40 = bitcast i8* bitcast (i32* @ul to i8*) to i32*
- %41 = cmpxchg i32* %40, i32 %37, i32 %39 monotonic monotonic
+ %pair41 = cmpxchg i32* %40, i32 %37, i32 %39 monotonic monotonic
+ %41 = extractvalue { i32, i1 } %pair41, 0
store i32 %41, i32* @ul, align 4
%42 = load i8* @uc, align 1
%43 = load i8* @sc, align 1
- %44 = cmpxchg i8* @sc, i8 %42, i8 %43 monotonic monotonic
+ %pair44 = cmpxchg i8* @sc, i8 %42, i8 %43 monotonic monotonic
+ %44 = extractvalue { i8, i1 } %pair44, 0
%45 = icmp eq i8 %44, %42
%46 = zext i1 %45 to i32
store i32 %46, i32* @ui, align 4
%47 = load i8* @uc, align 1
%48 = load i8* @sc, align 1
- %49 = cmpxchg i8* @uc, i8 %47, i8 %48 monotonic monotonic
+ %pair49 = cmpxchg i8* @uc, i8 %47, i8 %48 monotonic monotonic
+ %49 = extractvalue { i8, i1 } %pair49, 0
%50 = icmp eq i8 %49, %47
%51 = zext i1 %50 to i32
store i32 %51, i32* @ui, align 4
%54 = load i8* @sc, align 1
%55 = sext i8 %54 to i16
%56 = bitcast i8* bitcast (i16* @ss to i8*) to i16*
- %57 = cmpxchg i16* %56, i16 %53, i16 %55 monotonic monotonic
+ %pair57 = cmpxchg i16* %56, i16 %53, i16 %55 monotonic monotonic
+ %57 = extractvalue { i16, i1 } %pair57, 0
%58 = icmp eq i16 %57, %53
%59 = zext i1 %58 to i32
store i32 %59, i32* @ui, align 4
%62 = load i8* @sc, align 1
%63 = sext i8 %62 to i16
%64 = bitcast i8* bitcast (i16* @us to i8*) to i16*
- %65 = cmpxchg i16* %64, i16 %61, i16 %63 monotonic monotonic
+ %pair65 = cmpxchg i16* %64, i16 %61, i16 %63 monotonic monotonic
+ %65 = extractvalue { i16, i1 } %pair65, 0
%66 = icmp eq i16 %65, %61
%67 = zext i1 %66 to i32
store i32 %67, i32* @ui, align 4
%70 = load i8* @sc, align 1
%71 = sext i8 %70 to i32
%72 = bitcast i8* bitcast (i32* @si to i8*) to i32*
- %73 = cmpxchg i32* %72, i32 %69, i32 %71 monotonic monotonic
+ %pair73 = cmpxchg i32* %72, i32 %69, i32 %71 monotonic monotonic
+ %73 = extractvalue { i32, i1 } %pair73, 0
%74 = icmp eq i32 %73, %69
%75 = zext i1 %74 to i32
store i32 %75, i32* @ui, align 4
%78 = load i8* @sc, align 1
%79 = sext i8 %78 to i32
%80 = bitcast i8* bitcast (i32* @ui to i8*) to i32*
- %81 = cmpxchg i32* %80, i32 %77, i32 %79 monotonic monotonic
+ %pair81 = cmpxchg i32* %80, i32 %77, i32 %79 monotonic monotonic
+ %81 = extractvalue { i32, i1 } %pair81, 0
%82 = icmp eq i32 %81, %77
%83 = zext i1 %82 to i32
store i32 %83, i32* @ui, align 4
%86 = load i8* @sc, align 1
%87 = sext i8 %86 to i32
%88 = bitcast i8* bitcast (i32* @sl to i8*) to i32*
- %89 = cmpxchg i32* %88, i32 %85, i32 %87 monotonic monotonic
+ %pair89 = cmpxchg i32* %88, i32 %85, i32 %87 monotonic monotonic
+ %89 = extractvalue { i32, i1 } %pair89, 0
%90 = icmp eq i32 %89, %85
%91 = zext i1 %90 to i32
store i32 %91, i32* @ui, align 4
%94 = load i8* @sc, align 1
%95 = sext i8 %94 to i32
%96 = bitcast i8* bitcast (i32* @ul to i8*) to i32*
- %97 = cmpxchg i32* %96, i32 %93, i32 %95 monotonic monotonic
+ %pair97 = cmpxchg i32* %96, i32 %93, i32 %95 monotonic monotonic
+ %97 = extractvalue { i32, i1 } %pair97, 0
%98 = icmp eq i32 %97, %93
%99 = zext i1 %98 to i32
store i32 %99, i32* @ui, align 4
define i32 @exchange_and_cmp(i32* %mem) nounwind {
; CHECK-LABEL: exchange_and_cmp:
; CHECK: lwarx
- %tmp = cmpxchg i32* %mem, i32 0, i32 1 monotonic monotonic
+ %tmppair = cmpxchg i32* %mem, i32 0, i32 1 monotonic monotonic
+ %tmp = extractvalue { i32, i1 } %tmppair, 0
; CHECK: stwcx.
; CHECK: stwcx.
ret i32 %tmp
define i64 @exchange_and_cmp(i64* %mem) nounwind {
; CHECK-LABEL: exchange_and_cmp:
; CHECK: ldarx
- %tmp = cmpxchg i64* %mem, i64 0, i64 1 monotonic monotonic
+ %tmppair = cmpxchg i64* %mem, i64 0, i64 1 monotonic monotonic
+ %tmp = extractvalue { i64, i1 } %tmppair, 0
; CHECK: stdcx.
; CHECK: stdcx.
ret i64 %tmp
; SI: S_ENDPGM
define void @lds_atomic_cmpxchg_ret_i32_offset(i32 addrspace(1)* %out, i32 addrspace(3)* %ptr, i32 %swap) nounwind {
%gep = getelementptr i32 addrspace(3)* %ptr, i32 4
- %result = cmpxchg i32 addrspace(3)* %gep, i32 7, i32 %swap seq_cst monotonic
+ %pair = cmpxchg i32 addrspace(3)* %gep, i32 7, i32 %swap seq_cst monotonic
+ %result = extractvalue { i32, i1 } %pair, 0
store i32 %result, i32 addrspace(1)* %out, align 4
ret void
}
; SI: S_ENDPGM
define void @lds_atomic_cmpxchg_ret_i64_offset(i64 addrspace(1)* %out, i64 addrspace(3)* %ptr, i64 %swap) nounwind {
%gep = getelementptr i64 addrspace(3)* %ptr, i32 4
- %result = cmpxchg i64 addrspace(3)* %gep, i64 7, i64 %swap seq_cst monotonic
+ %pair = cmpxchg i64 addrspace(3)* %gep, i64 7, i64 %swap seq_cst monotonic
+ %result = extractvalue { i64, i1 } %pair, 0
store i64 %result, i64 addrspace(1)* %out, align 8
ret void
}
define i32 @test_cmpxchg_i32(i32 %a, i32* %ptr) {
entry:
- %b = cmpxchg i32* %ptr, i32 %a, i32 123 monotonic monotonic
+ %pair = cmpxchg i32* %ptr, i32 %a, i32 123 monotonic monotonic
+ %b = extractvalue { i32, i1 } %pair, 0
ret i32 %b
}
define i64 @test_cmpxchg_i64(i64 %a, i64* %ptr) {
entry:
- %b = cmpxchg i64* %ptr, i64 %a, i64 123 monotonic monotonic
+ %pair = cmpxchg i64* %ptr, i64 %a, i64 123 monotonic monotonic
+ %b = extractvalue { i64, i1 } %pair, 0
ret i64 %b
}
; CHECK-SHIFT: lcr [[NEGSHIFT:%r[1-9]+]], [[SHIFT]]
; CHECK-SHIFT: rll
; CHECK-SHIFT: rll {{%r[0-9]+}}, %r5, -8([[NEGSHIFT]])
- %res = cmpxchg i8 *%src, i8 %cmp, i8 %swap seq_cst seq_cst
+ %pair = cmpxchg i8 *%src, i8 %cmp, i8 %swap seq_cst seq_cst
+ %res = extractvalue { i8, i1 } %pair, 0
ret i8 %res
}
; CHECK-SHIFT: risbg
; CHECK-SHIFT: risbg [[SWAP]], {{%r[0-9]+}}, 32, 55, 0
; CHECK-SHIFT: br %r14
- %res = cmpxchg i8 *%src, i8 42, i8 88 seq_cst seq_cst
+ %pair = cmpxchg i8 *%src, i8 42, i8 88 seq_cst seq_cst
+ %res = extractvalue { i8, i1 } %pair, 0
ret i8 %res
}
; CHECK-SHIFT: lcr [[NEGSHIFT:%r[1-9]+]], [[SHIFT]]
; CHECK-SHIFT: rll
; CHECK-SHIFT: rll {{%r[0-9]+}}, %r5, -16([[NEGSHIFT]])
- %res = cmpxchg i16 *%src, i16 %cmp, i16 %swap seq_cst seq_cst
+ %pair = cmpxchg i16 *%src, i16 %cmp, i16 %swap seq_cst seq_cst
+ %res = extractvalue { i16, i1 } %pair, 0
ret i16 %res
}
; CHECK-SHIFT: risbg
; CHECK-SHIFT: risbg [[SWAP]], {{%r[0-9]+}}, 32, 47, 0
; CHECK-SHIFT: br %r14
- %res = cmpxchg i16 *%src, i16 42, i16 88 seq_cst seq_cst
+ %pair = cmpxchg i16 *%src, i16 42, i16 88 seq_cst seq_cst
+ %res = extractvalue { i16, i1 } %pair, 0
ret i16 %res
}
; CHECK-LABEL: f1:
; CHECK: cs %r2, %r3, 0(%r4)
; CHECK: br %r14
- %val = cmpxchg i32 *%src, i32 %cmp, i32 %swap seq_cst seq_cst
+ %pair = cmpxchg i32 *%src, i32 %cmp, i32 %swap seq_cst seq_cst
+ %val = extractvalue { i32, i1 } %pair, 0
ret i32 %val
}
; CHECK: cs %r2, %r3, 4092(%r4)
; CHECK: br %r14
%ptr = getelementptr i32 *%src, i64 1023
- %val = cmpxchg i32 *%ptr, i32 %cmp, i32 %swap seq_cst seq_cst
+ %pair = cmpxchg i32 *%ptr, i32 %cmp, i32 %swap seq_cst seq_cst
+ %val = extractvalue { i32, i1 } %pair, 0
ret i32 %val
}
; CHECK: csy %r2, %r3, 4096(%r4)
; CHECK: br %r14
%ptr = getelementptr i32 *%src, i64 1024
- %val = cmpxchg i32 *%ptr, i32 %cmp, i32 %swap seq_cst seq_cst
+ %pair = cmpxchg i32 *%ptr, i32 %cmp, i32 %swap seq_cst seq_cst
+ %val = extractvalue { i32, i1 } %pair, 0
ret i32 %val
}
; CHECK: csy %r2, %r3, 524284(%r4)
; CHECK: br %r14
%ptr = getelementptr i32 *%src, i64 131071
- %val = cmpxchg i32 *%ptr, i32 %cmp, i32 %swap seq_cst seq_cst
+ %pair = cmpxchg i32 *%ptr, i32 %cmp, i32 %swap seq_cst seq_cst
+ %val = extractvalue { i32, i1 } %pair, 0
ret i32 %val
}
; CHECK: cs %r2, %r3, 0(%r4)
; CHECK: br %r14
%ptr = getelementptr i32 *%src, i64 131072
- %val = cmpxchg i32 *%ptr, i32 %cmp, i32 %swap seq_cst seq_cst
+ %pair = cmpxchg i32 *%ptr, i32 %cmp, i32 %swap seq_cst seq_cst
+ %val = extractvalue { i32, i1 } %pair, 0
ret i32 %val
}
; CHECK: csy %r2, %r3, -4(%r4)
; CHECK: br %r14
%ptr = getelementptr i32 *%src, i64 -1
- %val = cmpxchg i32 *%ptr, i32 %cmp, i32 %swap seq_cst seq_cst
+ %pair = cmpxchg i32 *%ptr, i32 %cmp, i32 %swap seq_cst seq_cst
+ %val = extractvalue { i32, i1 } %pair, 0
ret i32 %val
}
; CHECK: csy %r2, %r3, -524288(%r4)
; CHECK: br %r14
%ptr = getelementptr i32 *%src, i64 -131072
- %val = cmpxchg i32 *%ptr, i32 %cmp, i32 %swap seq_cst seq_cst
+ %pair = cmpxchg i32 *%ptr, i32 %cmp, i32 %swap seq_cst seq_cst
+ %val = extractvalue { i32, i1 } %pair, 0
ret i32 %val
}
; CHECK: cs %r2, %r3, 0(%r4)
; CHECK: br %r14
%ptr = getelementptr i32 *%src, i64 -131073
- %val = cmpxchg i32 *%ptr, i32 %cmp, i32 %swap seq_cst seq_cst
+ %pair = cmpxchg i32 *%ptr, i32 %cmp, i32 %swap seq_cst seq_cst
+ %val = extractvalue { i32, i1 } %pair, 0
ret i32 %val
}
; CHECK: br %r14
%add1 = add i64 %src, %index
%ptr = inttoptr i64 %add1 to i32 *
- %val = cmpxchg i32 *%ptr, i32 %cmp, i32 %swap seq_cst seq_cst
+ %pair = cmpxchg i32 *%ptr, i32 %cmp, i32 %swap seq_cst seq_cst
+ %val = extractvalue { i32, i1 } %pair, 0
ret i32 %val
}
%add1 = add i64 %src, %index
%add2 = add i64 %add1, 4096
%ptr = inttoptr i64 %add2 to i32 *
- %val = cmpxchg i32 *%ptr, i32 %cmp, i32 %swap seq_cst seq_cst
+ %pair = cmpxchg i32 *%ptr, i32 %cmp, i32 %swap seq_cst seq_cst
+ %val = extractvalue { i32, i1 } %pair, 0
ret i32 %val
}
; CHECK: lhi %r2, 1001
; CHECK: cs %r2, %r3, 0(%r4)
; CHECK: br %r14
- %val = cmpxchg i32 *%ptr, i32 1001, i32 %swap seq_cst seq_cst
+ %pair = cmpxchg i32 *%ptr, i32 1001, i32 %swap seq_cst seq_cst
+ %val = extractvalue { i32, i1 } %pair, 0
ret i32 %val
}
; CHECK: lhi [[SWAP:%r[0-9]+]], 1002
; CHECK: cs %r2, [[SWAP]], 0(%r3)
; CHECK: br %r14
- %val = cmpxchg i32 *%ptr, i32 %cmp, i32 1002 seq_cst seq_cst
+ %pair = cmpxchg i32 *%ptr, i32 %cmp, i32 1002 seq_cst seq_cst
+ %val = extractvalue { i32, i1 } %pair, 0
ret i32 %val
}
; CHECK-LABEL: f1:
; CHECK: csg %r2, %r3, 0(%r4)
; CHECK: br %r14
- %val = cmpxchg i64 *%src, i64 %cmp, i64 %swap seq_cst seq_cst
+ %pairval = cmpxchg i64 *%src, i64 %cmp, i64 %swap seq_cst seq_cst
+ %val = extractvalue { i64, i1 } %pairval, 0
ret i64 %val
}
; CHECK: csg %r2, %r3, 524280(%r4)
; CHECK: br %r14
%ptr = getelementptr i64 *%src, i64 65535
- %val = cmpxchg i64 *%ptr, i64 %cmp, i64 %swap seq_cst seq_cst
+ %pairval = cmpxchg i64 *%ptr, i64 %cmp, i64 %swap seq_cst seq_cst
+ %val = extractvalue { i64, i1 } %pairval, 0
ret i64 %val
}
; CHECK: csg %r2, %r3, 0(%r4)
; CHECK: br %r14
%ptr = getelementptr i64 *%src, i64 65536
- %val = cmpxchg i64 *%ptr, i64 %cmp, i64 %swap seq_cst seq_cst
+ %pairval = cmpxchg i64 *%ptr, i64 %cmp, i64 %swap seq_cst seq_cst
+ %val = extractvalue { i64, i1 } %pairval, 0
ret i64 %val
}
; CHECK: csg %r2, %r3, -8(%r4)
; CHECK: br %r14
%ptr = getelementptr i64 *%src, i64 -1
- %val = cmpxchg i64 *%ptr, i64 %cmp, i64 %swap seq_cst seq_cst
+ %pairval = cmpxchg i64 *%ptr, i64 %cmp, i64 %swap seq_cst seq_cst
+ %val = extractvalue { i64, i1 } %pairval, 0
ret i64 %val
}
; CHECK: csg %r2, %r3, -524288(%r4)
; CHECK: br %r14
%ptr = getelementptr i64 *%src, i64 -65536
- %val = cmpxchg i64 *%ptr, i64 %cmp, i64 %swap seq_cst seq_cst
+ %pairval = cmpxchg i64 *%ptr, i64 %cmp, i64 %swap seq_cst seq_cst
+ %val = extractvalue { i64, i1 } %pairval, 0
ret i64 %val
}
; CHECK: csg %r2, %r3, 0(%r4)
; CHECK: br %r14
%ptr = getelementptr i64 *%src, i64 -65537
- %val = cmpxchg i64 *%ptr, i64 %cmp, i64 %swap seq_cst seq_cst
+ %pairval = cmpxchg i64 *%ptr, i64 %cmp, i64 %swap seq_cst seq_cst
+ %val = extractvalue { i64, i1 } %pairval, 0
ret i64 %val
}
; CHECK: br %r14
%add1 = add i64 %src, %index
%ptr = inttoptr i64 %add1 to i64 *
- %val = cmpxchg i64 *%ptr, i64 %cmp, i64 %swap seq_cst seq_cst
+ %pairval = cmpxchg i64 *%ptr, i64 %cmp, i64 %swap seq_cst seq_cst
+ %val = extractvalue { i64, i1 } %pairval, 0
ret i64 %val
}
; CHECK: lghi %r2, 1001
; CHECK: csg %r2, %r3, 0(%r4)
; CHECK: br %r14
- %val = cmpxchg i64 *%ptr, i64 1001, i64 %swap seq_cst seq_cst
+ %pairval = cmpxchg i64 *%ptr, i64 1001, i64 %swap seq_cst seq_cst
+ %val = extractvalue { i64, i1 } %pairval, 0
ret i64 %val
}
; CHECK: lghi [[SWAP:%r[0-9]+]], 1002
; CHECK: csg %r2, [[SWAP]], 0(%r3)
; CHECK: br %r14
- %val = cmpxchg i64 *%ptr, i64 %cmp, i64 1002 seq_cst seq_cst
+ %pairval = cmpxchg i64 *%ptr, i64 %cmp, i64 1002 seq_cst seq_cst
+ %val = extractvalue { i64, i1 } %pairval, 0
ret i64 %val
}
loop:
; CHECK: lock
; CHECK-NEXT: cmpxchg8b
- %r = cmpxchg i64* %ptr, i64 0, i64 1 monotonic monotonic
+ %pair = cmpxchg i64* %ptr, i64 0, i64 1 monotonic monotonic
+ %r = extractvalue { i64, i1 } %pair, 0
%stored1 = icmp eq i64 %r, 0
br i1 %stored1, label %loop, label %continue
continue:
%3 = zext i8 %2 to i32
%4 = trunc i32 %3 to i8
%5 = trunc i32 %1 to i8
- %6 = cmpxchg i8* @sc, i8 %4, i8 %5 monotonic monotonic
+ %pair6 = cmpxchg i8* @sc, i8 %4, i8 %5 monotonic monotonic
+ %6 = extractvalue { i8, i1 } %pair6, 0
store i8 %6, i8* @sc, align 1
%7 = load i8* @sc, align 1
%8 = zext i8 %7 to i32
%10 = zext i8 %9 to i32
%11 = trunc i32 %10 to i8
%12 = trunc i32 %8 to i8
- %13 = cmpxchg i8* @uc, i8 %11, i8 %12 monotonic monotonic
+ %pair13 = cmpxchg i8* @uc, i8 %11, i8 %12 monotonic monotonic
+ %13 = extractvalue { i8, i1 } %pair13, 0
store i8 %13, i8* @uc, align 1
%14 = load i8* @sc, align 1
%15 = sext i8 %14 to i16
%19 = bitcast i8* bitcast (i16* @ss to i8*) to i16*
%20 = trunc i32 %18 to i16
%21 = trunc i32 %16 to i16
- %22 = cmpxchg i16* %19, i16 %20, i16 %21 monotonic monotonic
+ %pair22 = cmpxchg i16* %19, i16 %20, i16 %21 monotonic monotonic
+ %22 = extractvalue { i16, i1 } %pair22, 0
store i16 %22, i16* @ss, align 2
%23 = load i8* @sc, align 1
%24 = sext i8 %23 to i16
%28 = bitcast i8* bitcast (i16* @us to i8*) to i16*
%29 = trunc i32 %27 to i16
%30 = trunc i32 %25 to i16
- %31 = cmpxchg i16* %28, i16 %29, i16 %30 monotonic monotonic
+ %pair31 = cmpxchg i16* %28, i16 %29, i16 %30 monotonic monotonic
+ %31 = extractvalue { i16, i1 } %pair31, 0
store i16 %31, i16* @us, align 2
%32 = load i8* @sc, align 1
%33 = sext i8 %32 to i32
%34 = load i8* @uc, align 1
%35 = zext i8 %34 to i32
%36 = bitcast i8* bitcast (i32* @si to i8*) to i32*
- %37 = cmpxchg i32* %36, i32 %35, i32 %33 monotonic monotonic
+ %pair37 = cmpxchg i32* %36, i32 %35, i32 %33 monotonic monotonic
+ %37 = extractvalue { i32, i1 } %pair37, 0
store i32 %37, i32* @si, align 4
%38 = load i8* @sc, align 1
%39 = sext i8 %38 to i32
%40 = load i8* @uc, align 1
%41 = zext i8 %40 to i32
%42 = bitcast i8* bitcast (i32* @ui to i8*) to i32*
- %43 = cmpxchg i32* %42, i32 %41, i32 %39 monotonic monotonic
+ %pair43 = cmpxchg i32* %42, i32 %41, i32 %39 monotonic monotonic
+ %43 = extractvalue { i32, i1 } %pair43, 0
store i32 %43, i32* @ui, align 4
%44 = load i8* @sc, align 1
%45 = sext i8 %44 to i64
%46 = load i8* @uc, align 1
%47 = zext i8 %46 to i64
%48 = bitcast i8* bitcast (i64* @sl to i8*) to i64*
- %49 = cmpxchg i64* %48, i64 %47, i64 %45 monotonic monotonic
+ %pair49 = cmpxchg i64* %48, i64 %47, i64 %45 monotonic monotonic
+ %49 = extractvalue { i64, i1 } %pair49, 0
store i64 %49, i64* @sl, align 8
%50 = load i8* @sc, align 1
%51 = sext i8 %50 to i64
%52 = load i8* @uc, align 1
%53 = zext i8 %52 to i64
%54 = bitcast i8* bitcast (i64* @ul to i8*) to i64*
- %55 = cmpxchg i64* %54, i64 %53, i64 %51 monotonic monotonic
+ %pair55 = cmpxchg i64* %54, i64 %53, i64 %51 monotonic monotonic
+ %55 = extractvalue { i64, i1 } %pair55, 0
store i64 %55, i64* @ul, align 8
%56 = load i8* @sc, align 1
%57 = sext i8 %56 to i64
%58 = load i8* @uc, align 1
%59 = zext i8 %58 to i64
%60 = bitcast i8* bitcast (i64* @sll to i8*) to i64*
- %61 = cmpxchg i64* %60, i64 %59, i64 %57 monotonic monotonic
+ %pair61 = cmpxchg i64* %60, i64 %59, i64 %57 monotonic monotonic
+ %61 = extractvalue { i64, i1 } %pair61, 0
store i64 %61, i64* @sll, align 8
%62 = load i8* @sc, align 1
%63 = sext i8 %62 to i64
%64 = load i8* @uc, align 1
%65 = zext i8 %64 to i64
%66 = bitcast i8* bitcast (i64* @ull to i8*) to i64*
- %67 = cmpxchg i64* %66, i64 %65, i64 %63 monotonic monotonic
+ %pair67 = cmpxchg i64* %66, i64 %65, i64 %63 monotonic monotonic
+ %67 = extractvalue { i64, i1 } %pair67, 0
store i64 %67, i64* @ull, align 8
%68 = load i8* @sc, align 1
%69 = zext i8 %68 to i32
%71 = zext i8 %70 to i32
%72 = trunc i32 %71 to i8
%73 = trunc i32 %69 to i8
- %74 = cmpxchg i8* @sc, i8 %72, i8 %73 monotonic monotonic
+ %pair74 = cmpxchg i8* @sc, i8 %72, i8 %73 monotonic monotonic
+ %74 = extractvalue { i8, i1 } %pair74, 0
%75 = icmp eq i8 %74, %72
%76 = zext i1 %75 to i8
%77 = zext i8 %76 to i32
%81 = zext i8 %80 to i32
%82 = trunc i32 %81 to i8
%83 = trunc i32 %79 to i8
- %84 = cmpxchg i8* @uc, i8 %82, i8 %83 monotonic monotonic
+ %pair84 = cmpxchg i8* @uc, i8 %82, i8 %83 monotonic monotonic
+ %84 = extractvalue { i8, i1 } %pair84, 0
%85 = icmp eq i8 %84, %82
%86 = zext i1 %85 to i8
%87 = zext i8 %86 to i32
%92 = zext i8 %91 to i32
%93 = trunc i32 %92 to i8
%94 = trunc i32 %90 to i8
- %95 = cmpxchg i8* bitcast (i16* @ss to i8*), i8 %93, i8 %94 monotonic monotonic
+ %pair95 = cmpxchg i8* bitcast (i16* @ss to i8*), i8 %93, i8 %94 monotonic monotonic
+ %95 = extractvalue { i8, i1 } %pair95, 0
%96 = icmp eq i8 %95, %93
%97 = zext i1 %96 to i8
%98 = zext i8 %97 to i32
%103 = zext i8 %102 to i32
%104 = trunc i32 %103 to i8
%105 = trunc i32 %101 to i8
- %106 = cmpxchg i8* bitcast (i16* @us to i8*), i8 %104, i8 %105 monotonic monotonic
+ %pair106 = cmpxchg i8* bitcast (i16* @us to i8*), i8 %104, i8 %105 monotonic monotonic
+ %106 = extractvalue { i8, i1 } %pair106, 0
%107 = icmp eq i8 %106, %104
%108 = zext i1 %107 to i8
%109 = zext i8 %108 to i32
%113 = zext i8 %112 to i32
%114 = trunc i32 %113 to i8
%115 = trunc i32 %111 to i8
- %116 = cmpxchg i8* bitcast (i32* @si to i8*), i8 %114, i8 %115 monotonic monotonic
+ %pair116 = cmpxchg i8* bitcast (i32* @si to i8*), i8 %114, i8 %115 monotonic monotonic
+ %116 = extractvalue { i8, i1 } %pair116, 0
%117 = icmp eq i8 %116, %114
%118 = zext i1 %117 to i8
%119 = zext i8 %118 to i32
%123 = zext i8 %122 to i32
%124 = trunc i32 %123 to i8
%125 = trunc i32 %121 to i8
- %126 = cmpxchg i8* bitcast (i32* @ui to i8*), i8 %124, i8 %125 monotonic monotonic
+ %pair126 = cmpxchg i8* bitcast (i32* @ui to i8*), i8 %124, i8 %125 monotonic monotonic
+ %126 = extractvalue { i8, i1 } %pair126, 0
%127 = icmp eq i8 %126, %124
%128 = zext i1 %127 to i8
%129 = zext i8 %128 to i32
%133 = zext i8 %132 to i64
%134 = trunc i64 %133 to i8
%135 = trunc i64 %131 to i8
- %136 = cmpxchg i8* bitcast (i64* @sl to i8*), i8 %134, i8 %135 monotonic monotonic
+ %pair136 = cmpxchg i8* bitcast (i64* @sl to i8*), i8 %134, i8 %135 monotonic monotonic
+ %136 = extractvalue { i8, i1 } %pair136, 0
%137 = icmp eq i8 %136, %134
%138 = zext i1 %137 to i8
%139 = zext i8 %138 to i32
%143 = zext i8 %142 to i64
%144 = trunc i64 %143 to i8
%145 = trunc i64 %141 to i8
- %146 = cmpxchg i8* bitcast (i64* @ul to i8*), i8 %144, i8 %145 monotonic monotonic
+ %pair146 = cmpxchg i8* bitcast (i64* @ul to i8*), i8 %144, i8 %145 monotonic monotonic
+ %146 = extractvalue { i8, i1 } %pair146, 0
%147 = icmp eq i8 %146, %144
%148 = zext i1 %147 to i8
%149 = zext i8 %148 to i32
%153 = zext i8 %152 to i64
%154 = trunc i64 %153 to i8
%155 = trunc i64 %151 to i8
- %156 = cmpxchg i8* bitcast (i64* @sll to i8*), i8 %154, i8 %155 monotonic monotonic
+ %pair156 = cmpxchg i8* bitcast (i64* @sll to i8*), i8 %154, i8 %155 monotonic monotonic
+ %156 = extractvalue { i8, i1 } %pair156, 0
%157 = icmp eq i8 %156, %154
%158 = zext i1 %157 to i8
%159 = zext i8 %158 to i32
%163 = zext i8 %162 to i64
%164 = trunc i64 %163 to i8
%165 = trunc i64 %161 to i8
- %166 = cmpxchg i8* bitcast (i64* @ull to i8*), i8 %164, i8 %165 monotonic monotonic
+ %pair166 = cmpxchg i8* bitcast (i64* @ull to i8*), i8 %164, i8 %165 monotonic monotonic
+ %166 = extractvalue { i8, i1 } %pair166, 0
%167 = icmp eq i8 %166, %164
%168 = zext i1 %167 to i8
%169 = zext i8 %168 to i32
%neg1 = sub i32 0, 10 ; <i32> [#uses=1]
; CHECK: lock
; CHECK: cmpxchgl
- %16 = cmpxchg i32* %val2, i32 %neg1, i32 1 monotonic monotonic
+ %pair16 = cmpxchg i32* %val2, i32 %neg1, i32 1 monotonic monotonic
+ %16 = extractvalue { i32, i1 } %pair16, 0
store i32 %16, i32* %old
; CHECK: lock
; CHECK: cmpxchgl
- %17 = cmpxchg i32* %val2, i32 1976, i32 1 monotonic monotonic
+ %pair17 = cmpxchg i32* %val2, i32 1976, i32 1 monotonic monotonic
+ %17 = extractvalue { i32, i1 } %pair17, 0
store i32 %17, i32* %old
; CHECK: movl [[R17atomic:.*]], %eax
; CHECK: movl $1401, %[[R17mask:[a-z]*]]
; CHECK: lock
; CHECK: cmpxchgl %{{.*}}, %gs:(%{{.*}})
- %0 = cmpxchg i32 addrspace(256)* %P, i32 0, i32 1 monotonic monotonic
+ %pair0 = cmpxchg i32 addrspace(256)* %P, i32 0, i32 1 monotonic monotonic
+ %0 = extractvalue { i32, i1 } %pair0, 0
ret void
}
define i32 @main() nounwind {
entry:
- %0 = cmpxchg i64* @val, i64 0, i64 1 monotonic monotonic
+ %t0 = cmpxchg i64* @val, i64 0, i64 1 monotonic monotonic
+ %0 = extractvalue { i64, i1 } %t0, 0
%1 = tail call i32 (i8*, ...)* @printf(i8* getelementptr ([7 x i8]* @"\01LC", i32 0, i64 0), i64 %0) nounwind
ret i32 0
}
define i32 @Cmpxchg(i32* %p, i32 %a, i32 %b) sanitize_memory {
entry:
- %0 = cmpxchg i32* %p, i32 %a, i32 %b seq_cst seq_cst
+ %pair = cmpxchg i32* %p, i32 %a, i32 %b seq_cst seq_cst
+ %0 = extractvalue { i32, i1 } %pair, 0
ret i32 %0
}
; CHECK: @Cmpxchg
-; CHECK: store i32 0,
+; CHECK: store { i32, i1 } zeroinitializer,
; CHECK: icmp
; CHECK: br
; CHECK: @__msan_warning
define i32 @CmpxchgMonotonic(i32* %p, i32 %a, i32 %b) sanitize_memory {
entry:
- %0 = cmpxchg i32* %p, i32 %a, i32 %b monotonic monotonic
+ %pair = cmpxchg i32* %p, i32 %a, i32 %b monotonic monotonic
+ %0 = extractvalue { i32, i1 } %pair, 0
ret i32 %0
}
; CHECK: @CmpxchgMonotonic
-; CHECK: store i32 0,
+; CHECK: store { i32, i1 } zeroinitializer,
; CHECK: icmp
; CHECK: br
; CHECK: @__msan_warning
; CHECK: br i1 [[TST]], label %[[LOOP]], label %[[BARRIER:.*]]
; CHECK: [[BARRIER]]:
+; CHECK: [[SUCCESS:%.*]] = phi i1 [ true, %[[TRY_STORE]] ], [ false, %[[LOOP]] ]
; CHECK: fence seq_cst
; CHECK: br label %[[DONE:.*]]
; CHECK: [[DONE]]:
; CHECK: ret i8 [[OLDVAL]]
- %old = cmpxchg i8* %ptr, i8 %desired, i8 %newval seq_cst seq_cst
+ %pairold = cmpxchg i8* %ptr, i8 %desired, i8 %newval seq_cst seq_cst
+ %old = extractvalue { i8, i1 } %pairold, 0
ret i8 %old
}
; CHECK: br label %[[DONE:.*]]
; CHECK: [[DONE]]:
+; CHECK: [[SUCCESS:%.*]] = phi i1 [ true, %[[BARRIER]] ], [ false, %[[LOOP]] ]
; CHECK: ret i16 [[OLDVAL]]
- %old = cmpxchg i16* %ptr, i16 %desired, i16 %newval seq_cst monotonic
+ %pairold = cmpxchg i16* %ptr, i16 %desired, i16 %newval seq_cst monotonic
+ %old = extractvalue { i16, i1 } %pairold, 0
ret i16 %old
}
; CHECK: br i1 [[TST]], label %[[LOOP]], label %[[BARRIER:.*]]
; CHECK: [[BARRIER]]:
+; CHECK: [[SUCCESS:%.*]] = phi i1 [ true, %[[TRY_STORE]] ], [ false, %[[LOOP]] ]
; CHECK: fence acquire
; CHECK: br label %[[DONE:.*]]
; CHECK: [[DONE]]:
; CHECK: ret i32 [[OLDVAL]]
- %old = cmpxchg i32* %ptr, i32 %desired, i32 %newval acquire acquire
+ %pairold = cmpxchg i32* %ptr, i32 %desired, i32 %newval acquire acquire
+ %old = extractvalue { i32, i1 } %pairold, 0
ret i32 %old
}
; CHECK: br label %[[DONE:.*]]
; CHECK: [[DONE]]:
+; CHECK: [[SUCCESS:%.*]] = phi i1 [ true, %[[BARRIER]] ], [ false, %[[LOOP]] ]
; CHECK: ret i64 [[OLDVAL]]
- %old = cmpxchg i64* %ptr, i64 %desired, i64 %newval monotonic monotonic
+ %pairold = cmpxchg i64* %ptr, i64 %desired, i64 %newval monotonic monotonic
+ %old = extractvalue { i64, i1 } %pairold, 0
ret i64 %old
}
\ No newline at end of file
; CHECK: br i1 [[TST]], label %[[LOOP]], label %[[BARRIER:.*]]
; CHECK: [[BARRIER]]:
+; CHECK: [[SUCCESS:%.*]] = phi i1 [ true, %[[TRY_STORE]] ], [ false, %[[LOOP]] ]
; CHECK-NOT: fence
; CHECK: br label %[[DONE:.*]]
; CHECK: [[DONE]]:
; CHECK: ret i8 [[OLDVAL]]
- %old = cmpxchg i8* %ptr, i8 %desired, i8 %newval seq_cst seq_cst
+ %pairold = cmpxchg i8* %ptr, i8 %desired, i8 %newval seq_cst seq_cst
+ %old = extractvalue { i8, i1 } %pairold, 0
ret i8 %old
}
; CHECK: br label %[[DONE:.*]]
; CHECK: [[DONE]]:
+; CHECK: [[SUCCESS:%.*]] = phi i1 [ true, %[[BARRIER]] ], [ false, %[[LOOP]] ]
; CHECK: ret i16 [[OLDVAL]]
- %old = cmpxchg i16* %ptr, i16 %desired, i16 %newval seq_cst monotonic
+ %pairold = cmpxchg i16* %ptr, i16 %desired, i16 %newval seq_cst monotonic
+ %old = extractvalue { i16, i1 } %pairold, 0
ret i16 %old
}
; CHECK: br i1 [[TST]], label %[[LOOP]], label %[[BARRIER:.*]]
; CHECK: [[BARRIER]]:
+; CHECK: [[SUCCESS:%.*]] = phi i1 [ true, %[[TRY_STORE]] ], [ false, %[[LOOP]] ]
; CHECK-NOT: fence
; CHECK: br label %[[DONE:.*]]
; CHECK: [[DONE]]:
; CHECK: ret i32 [[OLDVAL]]
- %old = cmpxchg i32* %ptr, i32 %desired, i32 %newval acquire acquire
+ %pairold = cmpxchg i32* %ptr, i32 %desired, i32 %newval acquire acquire
+ %old = extractvalue { i32, i1 } %pairold, 0
ret i32 %old
}
; CHECK: [[DONE]]:
; CHECK: ret i64 [[OLDVAL]]
- %old = cmpxchg i64* %ptr, i64 %desired, i64 %newval monotonic monotonic
+ %pairold = cmpxchg i64* %ptr, i64 %desired, i64 %newval monotonic monotonic
+ %old = extractvalue { i64, i1 } %pairold, 0
ret i64 %old
}
\ No newline at end of file
define i8 @cmpswap() {
; CHECK-LABEL: @cmpswap(
%i = alloca i8
- %j = cmpxchg i8* %i, i8 0, i8 42 monotonic monotonic
-; CHECK: [[INST:%[a-z0-9]+]] = load
-; CHECK-NEXT: icmp
-; CHECK-NEXT: select
-; CHECK-NEXT: store
+ %pair = cmpxchg i8* %i, i8 0, i8 42 monotonic monotonic
+ %j = extractvalue { i8, i1 } %pair, 0
+; CHECK: [[OLDVAL:%[a-z0-9]+]] = load i8* [[ADDR:%[a-z0-9]+]]
+; CHECK-NEXT: [[SAME:%[a-z0-9]+]] = icmp eq i8 [[OLDVAL]], 0
+; CHECK-NEXT: [[TO_STORE:%[a-z0-9]+]] = select i1 [[SAME]], i8 42, i8 [[OLDVAL]]
+; CHECK-NEXT: store i8 [[TO_STORE]], i8* [[ADDR]]
+; CHECK-NEXT: [[TMP:%[a-z0-9]+]] = insertvalue { i8, i1 } undef, i8 [[OLDVAL]], 0
+; CHECK-NEXT: [[RES:%[a-z0-9]+]] = insertvalue { i8, i1 } [[TMP]], i1 [[SAME]], 1
+; CHECK-NEXT: [[VAL:%[a-z0-9]+]] = extractvalue { i8, i1 } [[RES]], 0
ret i8 %j
-; CHECK: ret i8 [[INST]]
+; CHECK: ret i8 [[VAL]]
}
+
define i8 @swap() {
; CHECK-LABEL: @swap(
%i = alloca i8
projects / oota-llvm.git / commitdiff